Uh oh!
There was an error while loading. Please reload this page.
feat(AzureStorage): support orchestrator-only and activity-only worke… - #1375
Conversation
…rs via WorkerDispatchMode Add a WorkerDispatchMode setting (Both/Orchestrator/Activity) so a worker can run only orchestrations+entities or only activities. Activity mode skips control-queue partition leasing; Orchestrator mode skips the work-item queue. TaskHubWorker starts only the enabled dispatchers via the existing dispatcher-count capabilities. Default Both preserves current behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Adds a decoupled “worker dispatch mode” capability for the Azure Storage backend so a worker can be configured to dispatch only orchestrations/entities, only activities, or both—enabling independent scaling of orchestration vs. activity workers on the same task hub.
Changes:
- Introduces
WorkerDispatchModeand a correspondingAzureStorageOrchestrationServiceSettings.WorkerDispatchModesetting (defaultBoth). - Makes
AzureStorageOrchestrationServicedisable dispatcher counts and skip control-queue partition leasing for activity-only workers, with additional “safety net” backoff in fetch loops. - Updates
TaskHubWorkerto start only the dispatchers enabled by the backend’s dispatcher counts and adds a new Azurite-based decoupled worker test.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs | Adds unit + Azurite integration tests for orchestrator-only + activity-only cooperation on the same task hub. |
| src/DurableTask.Core/TaskHubWorker.cs | Gates dispatcher creation/start/stop based on dispatcher counts to support “disabled” dispatch types. |
| src/DurableTask.AzureStorage/WorkerDispatchMode.cs | Defines the new dispatch mode enum (Both/Orchestrator/Activity). |
| src/DurableTask.AzureStorage/AzureStorageOrchestrationServiceSettings.cs | Exposes WorkerDispatchMode setting with default Both. |
| src/DurableTask.AzureStorage/AzureStorageOrchestrationService.cs | Implements mode behavior via dispatcher counts, skips app lease manager in activity-only mode, and adds backoff safety nets. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
Test/DurableTask.AzureStorage.Tests/DecoupledWorkerDispatchTests.cs:24
- This test file is under
Test/(capital T), but the repository’s solution references the Azure Storage test project undertest/DurableTask.AzureStorage.Tests(lowercase). There is noTest/DurableTask.AzureStorage.Tests/*.csproj, so this test will not be compiled or executed in CI, meaning it won’t validate the new dispatch-mode behavior.
namespace DurableTask.AzureStorage.Tests
{
using DurableTask.AzureStorage;
using DurableTask.Core;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
using System.Linq;
using System.Threading.Tasks;
[TestClass]
public class DecoupledWorkerDispatchTests
src/DurableTask.Core/TaskHubWorker.cs:333
TaskHubWorkernow conditionally creates the orchestration/activity dispatchers based on dispatcher counts. This meansTaskOrchestrationDispatcher/TaskActivityDispatchercan remainnulleven afterStartAsync(), which is a behavioral change in the public API surface and can causeNullReferenceExceptions for consumers that configure dispatcher counts to 0 but still access these properties (e.g., to setIncludeDetails). SinceWorkItemDispatcheralready supportsDispatcherCount = 0(it simply starts zero loops), consider always constructing these dispatcher objects and only gating theirStartAsync()calls.
if (this.dispatchOrchestrations)
{
this.orchestrationDispatcher = new TaskOrchestrationDispatcher(
this.orchestrationService,
this.orchestrationManager,
this.orchestrationDispatchPipeline,
this.logHelper,
this.ErrorPropagationMode,
this.versioningSettings,
this.ExceptionPropertiesProvider);
}
if (this.dispatchActivities)
{
this.activityDispatcher = new TaskActivityDispatcher(
this.orchestrationService,
this.activityManager,
this.activityDispatchPipeline,
this.logHelper,
this.ErrorPropagationMode,
this.ExceptionPropertiesProvider);
}
Uh oh!
There was an error while loading. Please reload this page.
- Move DecoupledWorkerDispatchTests.cs from Test/ to test/ so the solution's test project compiles and runs it in CI. - Use TestHelpers.GetTestStorageAccountConnectionString() and GetTestTaskHubName() instead of a hard-coded UseDevelopmentStorage=true connection string and a fixed task hub name, honoring existing DurableTaskTest* configuration. - TaskHubWorker: always construct the orchestration/activity dispatchers and only gate their StartAsync() calls, so the public dispatcher properties stay non-null for callers that configure them post-start (e.g. IncludeDetails); a disabled dispatcher (count 0) still starts zero fetch loops. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
bb02231 to
8e6740dCompareUh oh!
There was an error while loading. Please reload this page.
…aching in ctor Addresses review feedback: TaskHubWorker previously cached TaskOrchestrationDispatcherCount/TaskActivityDispatcherCount in the constructor, which could go stale if a backend derives those counts from mutable settings changed between construction and StartAsync. The counts are now read inside StartAsync just before starting the dispatchers, so a non-zero count is always honored. Restores the constructor to its original form. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6d5f4a2b-07b2-4c3e-a250-378aa68aa065
| // Symmetric with StartAsync: activity-only workers never started the partition manager. | ||
| if (this.settings.WorkerDispatchMode != WorkerDispatchMode.Activity) | ||
| { | ||
| await this.appLeaseManager.StopAsync(); | ||
| } |
Add a WorkerDispatchMode setting (Both/Orchestrator/Activity) so a worker can run only orchestrations+entities or only activities. Activity mode skips control-queue partition leasing; Orchestrator mode skips the work-item queue. TaskHubWorker starts only the enabled dispatchers via the existing dispatcher-count capabilities. Default Both preserves current behavior.