Uh oh!
There was an error while loading. Please reload this page.
Add scheduled tasks feature and delayed entity signals - #160
Conversation
Add a recurring schedule feature (durabletask.scheduled) for parity with durabletask-dotnet, built on durable entities and a helper orchestrator. Enable it on a worker with configure_scheduled_tasks(worker) and manage schedules from the client via ScheduledTaskClient / ScheduleClient (create, describe, list, update, pause, resume, delete). To support the schedule entity's self-rearming, add an optional signal_time parameter to entity, orchestration-context, and client signal_entity methods, and make the in-memory backend honor delayed (scheduled) entity signals. Includes unit tests, in-memory E2E tests, and live DTS E2E tests for both the schedule feature and delayed signals, plus an example and changelog entries.
There was a problem hiding this comment.
Pull request overview
This PR introduces a new scheduled tasks capability (durabletask.scheduled) to the core Durable Task Python SDK, implemented on top of durable entities plus a helper orchestrator. As a prerequisite, it also adds delayed (scheduled) entity signals end-to-end (entity APIs, orchestration context, client APIs, and the in-memory test backend) to support time-based schedule re-arming.
Changes:
- Added the
durabletask.scheduledpackage (schedule entity, client surface, models/DTOs, transition rules, registration helper, and error types). - Added a
signal_timeparameter across entity/orchestration/clientsignal_entityAPIs and mapped it to the protobufscheduledTimefield. - Updated the in-memory backend to defer entity operations scheduled in the future; added unit/E2E coverage plus a runnable example and changelog entry.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/durabletask/scheduled/test_transitions.py | Unit tests for schedule status transition rules. |
| tests/durabletask/scheduled/test_scheduled_e2e.py | In-memory E2E coverage for schedule lifecycle and runs. |
| tests/durabletask/scheduled/test_schedule_entity.py | Unit tests for schedule entity behavior/actions via executor harness. |
| tests/durabletask/scheduled/test_models.py | Unit tests for scheduled-task models, validation, and serialization. |
| tests/durabletask/scheduled/init.py | Test package init. |
| tests/durabletask/entities/test_delayed_signals_e2e.py | In-memory E2E coverage for delayed entity signals (client + orchestrator). |
| tests/durabletask-azuremanaged/scheduled/test_dts_scheduled_e2e.py | Live DTS E2E coverage for scheduled tasks. |
| tests/durabletask-azuremanaged/scheduled/init.py | Azuremanaged test package init. |
| tests/durabletask-azuremanaged/entities/test_dts_delayed_signals_e2e.py | Live DTS E2E coverage for delayed entity signals. |
| examples/scheduled_tasks.py | End-to-end sample showing schedule creation and management. |
| durabletask/worker.py | Extends orchestration-context signal_entity plumbing to accept signal_time. |
| durabletask/testing/in_memory_backend.py | Implements delayed entity operation delivery in the in-memory backend. |
| durabletask/task.py | Updates orchestration-context API contract/docs to include signal_time. |
| durabletask/scheduled/transitions.py | Defines valid schedule state transitions keyed by operation. |
| durabletask/scheduled/schedule_status.py | Introduces ScheduleStatus enum. |
| durabletask/scheduled/schedule_entity.py | Implements the schedule entity state machine and re-arming via delayed signals. |
| durabletask/scheduled/registration.py | Adds configure_scheduled_tasks(worker) registration helper. |
| durabletask/scheduled/orchestrator.py | Adds helper orchestrator for “awaitable” schedule entity operations. |
| durabletask/scheduled/models.py | Adds public DTOs/options plus internal config/state serialization. |
| durabletask/scheduled/exceptions.py | Adds scheduled-task exception types. |
| durabletask/scheduled/client.py | Adds ScheduledTaskClient and per-schedule ScheduleClient. |
| durabletask/scheduled/init.py | Exposes the scheduled tasks public surface. |
| durabletask/internal/helpers.py | Adds scheduled time support to entity-signal orchestrator actions. |
| durabletask/internal/client_helpers.py | Adds scheduled time support to client SignalEntityRequest. |
| durabletask/entities/entity_context.py | Adds signal_time to entity-side signal_entity API and action emission. |
| durabletask/entities/durable_entity.py | Adds signal_time to DurableEntity.signal_entity facade. |
| durabletask/client.py | Adds signal_time to sync + async client signal_entity APIs. |
| CHANGELOG.md | Documents the new scheduled tasks feature and delayed entity signals. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…uled-tasks # Conflicts: # CHANGELOG.md # durabletask/client.py # durabletask/entities/entity_context.py # durabletask/internal/client_helpers.py
The JSON serializer encoded dataclasses via dataclasses.asdict before ever checking for a to_json hook, so a dataclass could not override its own serialization -- a problem for dataclasses whose fields are not JSON-native (e.g. timedelta/datetime). The read path already consults from_json before its dataclass branch; this makes the write path symmetric by checking the to_json hook first and falling back to asdict/SimpleNamespace.
With the serializer now honoring the to_json hook for dataclasses, the schedule option types expose to_json/from_json instead of bespoke to_dict/from_dict. The schedule entity operations can again annotate their input as the option dataclass and let the worker reconstruct it via the from_json hook, removing the manual _coerce_options shim and the Any-typed parameter workaround.
Bernd Verst (berndverst)
left a comment
There was a problem hiding this comment.
Review summary
Thanks andystaples — this is a solid, well-tested feature that mirrors the .NET ScheduledTasks design closely: the Schedule entity state machine, the execution-token mechanism that cancels stale timer signals, and the per-operation orchestrator so client writes can await entity completion. Lint is clean and the scheduled + delayed-signal unit/in-memory-E2E tests pass locally (48 passed).
The recent commits are good improvements: making ScheduleState.from_json converter-free means state round-trips under any code path (not just the worker's threaded converter), and signal_time -> scheduledTime is now correctly threaded through both the client request builder and the orchestrator-action builder (these previously hardcoded scheduledTime=None).
A few things I'd like addressed or explicitly deferred (details inline):
Recommend before merge
- Naive/aware
datetimecrash inScheduleClient._matches_filter: comparing an awareschedule_created_atagainst a possibly-naivecreated_from/created_toraisesTypeError. Cheap to fix + test. - Missing
WORKER_CAPABILITY_SCHEDULED_TASKSadvertisement: .NET advertises this capability; Python defines it in the protobuf but never sends it. Either wire it throughconfigure_scheduled_tasks/worker.py, or confirm the backend doesn't gate on it.
Reasonable follow-ups (these match current .NET behavior)
- Typed schedule exceptions flattened to
RuntimeErroracross the orchestration boundary. list_schedulesapplies status/created filters client-side, so pages may be underfilled.- Exclusive
created_from/created_tobounds (consider inclusive). - Cross-SDK entity-name casing (
@schedule@vs@Schedule@). - No async client surface (the SDK already has an async client; .NET offers async).
Overall: mergeable with minor fixes — I'd gate on the datetime fix and a decision on the capability flag; the rest can be tracked as follow-ups. Nice work.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
…solidation - Advertise WORKER_CAPABILITY_SCHEDULED_TASKS from configure_scheduled_tasks via a new TaskHubGrpcWorker.add_capability(), mirroring .NET's UseScheduledTasks (currently inert against the DTS backend, which only gates on HistoryStreaming). - Fix naive-vs-aware datetime TypeError in schedule list filtering: ScheduleQuery normalizes its created_from/created_to bounds to aware UTC, and the client filter defensively normalizes the stored timestamp. Bounds remain exclusive to match the .NET ScheduledTasks implementation. - Consolidate the duplicated _ensure_aware helpers into a single public helpers.ensure_aware, removing the cross-module private-usage warning. - Document that list_schedules applies status/created filters client-side, so pages may be underfilled (matches .NET). - Add unit tests for query normalization, exclusive filter bounds, and the scheduled-tasks capability advertisement.
Uh oh!
There was an error while loading. Please reload this page.
Summary
Adds a scheduled tasks feature (
durabletask.scheduled) to the Python SDK,bringing it to parity with the
Microsoft.DurableTask.ScheduledTaskspackage indurabletask-dotnet. A schedule periodically starts a target orchestration based
on a configurable interval and time window. The feature is built entirely on top
of durable entities plus a small helper orchestrator — no new sidecar/protocol
requirements.
To make schedules work, this PR also adds delayed (scheduled) entity signals
across the entity, orchestration-context, and client
signal_entityAPIs, andteaches the in-memory test backend to honor them.
What's new
durabletask.scheduledpackageconfigure_scheduled_tasks(worker)— registers the schedule entity andoperation orchestrator on a worker (parity with .NET's
UseScheduledTasks).ScheduledTaskClient—create_schedule,get_schedule,get_schedule_client,list_schedules.ScheduleClient— per-schedulecreate,describe,update,pause,resume,delete.ScheduleCreationOptions,ScheduleUpdateOptions,ScheduleDescription,ScheduleQuery,ScheduleStatus.ScheduleNotFoundError,ScheduleInvalidTransitionError,ScheduleClientValidationError.Schedules support
interval(≥ 1s),start_at,end_at, andstart_immediately_if_late, with a state machine (Uninitialized→Active⇄
Paused) and an execution-token mechanism that cancels stale timer signals,mirroring the .NET design.
Delayed entity signals (prerequisite)
signal_timeparameter to:EntityContext.signal_entity/DurableEntity.signal_entityOrchestrationContext.signal_entitysignal_entity(sync and async)scheduledTimefield. The public parametername
signal_timematches the .NET public API (SignalEntityOptions.SignalTime).scheduledTimeis inthe future (via a background timer thread), on both the client-signal and
orchestration-emitted-signal paths.
Tests
and state/options serialization round-trips.
prefix/status filters, delete; plus delayed-signal deferral via client and
orchestrator.
pytest.mark.dts): scheduled-task lifecycle and delayedsignals, verified against the DTS emulator.
Docs & examples
examples/scheduled_tasks.py.CHANGELOG.mdentries under Unreleased.Notes
durabletaskpackage (not a separate distribution as in.NET) for simpler installation and testing.
schedule, so list filtering usesthe
@schedule@<prefix>instance-ID prefix.