Move reconciliation to the durable Yieldstar runtime

.changeset/reconciler.md

@@ -6,9 +6,6 @@
 "@notation/reconciler": minor
 "@notation/resource": minor
 "@notation/state": minor
-"@notation/state-sqlite": minor
 ---
 
-Add the reconciler API, versioned event streams, renewable mutation leases,
-SQLite state, backend-neutral dashboard state, and compiled infrastructure
-graphs.
+Add durable Yieldstar 0.5.0 deploy and destroy workflows, a resident Node SQLite runtime for CLI execution, versioned event streams, backend-neutral dashboard state, and compiled infrastructure graphs.

docs/cli/dashboard.md

@@ -1,21 +1,13 @@
 # notation dashboard
 
 ```sh
-notation dashboard
+notation dashboard <entryPoint>
 ```
 
-Starts a local web dashboard for observing deployment state.
+Starts a local web dashboard for observing the deployment's resource state.
 
 ```sh
-notation dashboard
+notation dashboard infra/api.ts
 ```
 
-The dashboard uses the same state backend as deploy and destroy. Set
-`NOTATION_STATE_PATH` to select SQLite:
-
-```sh
-NOTATION_STATE_PATH=.notation/state.db notation dashboard
-```
-
-The server reads through `StateBackend`, so file and SQLite state produce the same
-dashboard payload.
+The dashboard reads `.notation/workflows.db`, the same database used by deploy, destroy, and plan. Set `NOTATION_DATABASE_PATH` to choose another SQLite database path.

docs/cli/deploy.md

@@ -4,40 +4,44 @@
 notation deploy <entryPoint>
 ```
 
-Compiles and deploys the stack to AWS.
+Compiles the stack and runs a durable deploy.
 
 ```sh
 notation deploy infra/api.ts
 ```
 
 ## Event stream
 
-`--json` writes versioned reconciler events to stdout as newline-delimited JSON. Build
-output and diagnostics move to stderr.
+`--json` writes versioned reconciler events to stdout as newline-delimited JSON. Build output, the execution ID, and diagnostics move to stderr.
 
 ```sh
 notation deploy infra/api.ts --json > deploy.ndjson
 ```
 
+## Durable execution
+
+The command prints its execution ID before starting provider work. If the process crashes, resume the same execution with that ID:
+
+```sh
+notation deploy infra/api.ts --execution-id <id>
+```
+
+Do not reuse a completed execution ID for a new deploy or for destroy.
+
+Retryable provider conditions and consistency reads suspend on durable SQLite timers. The CLI stays resident until the scheduler wakes the execution and the workflow completes. Provider results are replayed after their heap checkpoint, but a crash after the provider accepts a create or update and before that checkpoint repeats the call, so provider mutations must be idempotent. Reconciler event consumers must tolerate the equivalent duplicate-delivery window.
+
 ## What happens
 
-1. **Compile** – esbuild compiles infra and runtime modules to `dist/`.
+1. **Compile** – esbuild compiles infrastructure and runtime modules to `dist/`.
 
-2. **Build resource graph** – imports the compiled output and collects the declared resources.
+2. **Build resource graph** – the CLI imports the compiled output in-process and collects declared resources.
 
-3. **Reconcile** – the reconciler compares desired state (graph) against current state (`.notation/state.json`):
- - New resources → **create**
- - Changed params → **update**
- - No changes → **noop**
- - Orphaned resources (in state but not in graph) → **delete**
+3. **Order dependencies** – dependency levels run in topological order.
 
-4. **Topological deployment** – resources deploy in dependency order (levels). Resources at the same level deploy concurrently.
+4. **Reconcile** – Notation compares desired resources with Yieldstar stores, then creates, updates, recreates, or leaves each resource unchanged.
 
-5. **Drift detection** – enabled by default. Reads actual AWS state and compares against stored state. If drifted, Notation updates to match your definition.
+5. **Detect drift** – unchanged resources are read from the provider and repaired when their remote state differs.
 
-State is persisted to `.notation/state.json` after each operation. Set
-`NOTATION_STATE_PATH` to a path ending in `.db` or `.sqlite` to use SQLite:
+6. **Delete orphans** – persisted resources absent from the graph are deleted when their resource type is registered.
 
-```sh
-NOTATION_STATE_PATH=.notation/state.db notation deploy infra/api.ts
-```
+State, step results, timers, queued tasks, and resource stores are persisted to `.notation/workflows.db`. Set `NOTATION_DATABASE_PATH` to choose another SQLite database path.

docs/cli/destroy.md

@@ -4,7 +4,7 @@
 notation destroy <entryPoint>
 ```
 
-Removes all resources in the stack. Tears down runs in reverse dependency order, so routes are removed before APIs and Lambdas before IAM roles etc.
+Compiles the application and runs a durable destroy. Resources are removed in reverse dependency order, then registered persisted orphans are removed.
 
 ```sh
 notation destroy infra/api.ts
@@ -15,3 +15,13 @@ notation destroy infra/api.ts
 ```sh
 notation destroy infra/api.ts --json > destroy.ndjson
 ```
+
+The command prints its execution ID. Resume a crashed destroy with the same ID so checkpointed work can be replayed:
+
+```sh
+notation destroy infra/api.ts --execution-id <id>
+```
+
+Retryable deletes suspend on durable SQLite timers. Resource state is removed only after the provider delete succeeds or reports that the resource is already absent.
+
+The crash-window contract described under [notation deploy](./deploy.md#durable-execution) applies to deletes as well.

docs/internals/reconciler.md

@@ -1,115 +1,52 @@
 # Reconciler
 
-The reconciler runs deployment operations to transition infrastructure from its current state to the state defined in the project.
-
-Source: `@notation/reconciler`
+The reconciler expresses deployment and destruction as Yieldstar async generators. Notation owns desired-state decisions and provider lifecycle; the caller's Yieldstar runtime owns durable execution, waiting, and shared state.
 
 ## Deploy flow
 
-```ts [packages/reconciler/src/index.ts]
-const reconciler = new Reconciler({ state, registry, emit });
-await reconciler.deploy(resources, { dryRun, driftDetection });
-```
-
-The reconciler walks the resource graph and, for each resource, determines an action:
-
-| Condition | Decision |
-| --------------------------------------------- | ------------------ |
-| Not in state | **create** |
-| In state, params changed | **update** |
-| In state, params unchanged, no drift | **noop** |
-| In state, but deleted from AWS | **drift-recreate** |
-| In state, AWS state differs from stored state | **drift-update** |
-| In state, not in graph (orphan) | **delete** |
-
-The `dryRun` flag runs the full diffing pipeline without executing any operations, so you can preview what a deploy would do.
-
-## Topological ordering
+`deploy` takes the deployment hold, walks dependency levels in order, decides an action for every resource, executes provider calls as durable steps, persists the result in a resource store, and deletes registered orphans.
 
-Resources are deployed in dependency order using `buildResourceDepthLevels()`. This function partitions the resource graph into levels – each level contains resources whose dependencies have all been satisfied by previous levels.
+| Condition | Decision |
+| --- | --- |
+| Not in state | **create** |
+| In state, params changed | **update** |
+| In state, params unchanged, no drift | **noop** |
+| In state, but deleted from the provider | **drift-recreate** |
+| In state, provider state differs from stored state | **drift-update** |
+| In state, not in graph | **delete-orphan** |
 
-```
-Level 0: IAM Role, CloudWatch LogGroup
-Level 1: Lambda Function (depends on Role, LogGroup)
-Level 2: API Gateway Integration (depends on Lambda)
-Level 3: API Gateway Route (depends on Integration)
-```
+Dry-run deploy performs decisions and emits lifecycle events without provider mutations or state mutations. When drift detection is enabled, it can still call provider read operations to decide whether a nominal noop has drifted.
 
-Resources within a level deploy concurrently, so independent resources like the IAM Role and LogGroup above are provisioned in parallel. Dependent resources wait for their dependencies.
+## Destroy flow
 
-Destroy operates in reverse order with dependents getting removed before their dependencies.
+`destroy` is a first-class durable operation. It takes the same deployment hold as deploy, deletes desired resources in reverse dependency order, deletes registered persisted orphans, and conditionally removes each resource store only after the provider delete succeeds or reports that the resource is already absent.
 
-### Cycle detection
+## Waiting and replay
 
-Cycle detection is built in. If resources form a circular dependency, the build fails with:
+Provider calls are stable durable steps, but provider acknowledgement and the Yieldstar heap checkpoint are not atomic. If the process crashes between them, replay repeats the call, so provider create, update, and delete operations must be idempotent. Event subscribers must likewise tolerate duplicate delivery when a crash occurs before the event checkpoint.
 
-```
-Resource dependency cycle detected
-```
+A resource operation throws `ResourceOperationPendingError` when it has not finished. The error gives the reconciler a delay and optional callback context. The runtime stores the context, waits without keeping the process busy, and calls the same operation again. See [Operation errors](./resource.md#operation-errors) for the complete API.
 
-This catches configuration errors before any cloud operations are attempted.
+Each attempt, delay, event, state read, state write, and hold change has a stable step key. A resumed execution must use the same execution ID. A new deploy or destroy must use a new execution ID.
 
-## Drift detection
+## State and the deployment hold
 
-Drift detection is enabled by default. After confirming no local changes to a resource, the reconciler reads the resource's current state from AWS (via the resource's `read()` operation) and diffs it against stored state.
+Each resource is stored under `notation/resource-state` with a deployment-scoped ID. Conditional updates and deletes compare the snapshot's UUIDv7 `instanceId` and version, so a stale execution cannot modify a deleted and recreated store.
 
-If AWS has drifted (e.g. someone changed a Lambda timeout in the console, or an IAM policy was modified by another tool), Notation updates the resource to match the canoncial definition in the source code.
+Deploy and destroy take an exclusive hold on the deployment through one `notation/deployment-hold` store per deployment. `store.take` suspends a competing execution as a durable waiter and wakes it after the holder releases. A waiter that finds the hold already taken when it inspects it emits `reconciler.hold.waiting` naming the holding execution ID, so a wait behind a crashed execution is visible instead of silent; a holder that appears only between that inspection and the `take` suspends the waiter without the event.
 
-Properties marked as `volatile` in the schema (like `LastModified` timestamps) are excluded from drift comparison.
+A failed or suspended execution keeps its hold, which is what makes resuming it safe. The hold of an execution that will never be resumed is cleared with `clearDeploymentHold` from `@notation/reconciler/durable` — the only supported way out of that state.
 
 ## Events
 
-The reconciler emits events at each step of an operation's lifecycle. The default `createConsoleReconcilerSubscriber()` logs these to the console with formatted output.
+The durable workflows emit these events:
 
 | Event | When |
 | ------------------------------------ | --------------------------------------------------- |
 | `reconciler.deploy.decision` | After deciding what action to take for a resource |
 | `reconciler.drift.detected` | When drift is found between stored and actual state |
 | `reconciler.operation.lifecycle` | When an operation starts, finishes, skips, or fails |
+| `reconciler.hold.waiting` | When another execution holds the deployment |
 | `reconciler.orphan-deletion.skipped` | When no registered class can delete an orphan |
 
-Lifecycle events contain the operation (`create`, `read`, `update`, or `delete`) and its
-status (`start`, `success`, `error`, `skip`, or `dry-run`). Events carry the resource ID,
-type, and relevant diff or error details.
-
-## Operations
-
-Each CRUD operation is implemented as an async generator with retry support:
-
-- **`createResourceOperation`** – creates the resource, reads back its state, persists to state backend
-- **`updateResourceOperation`** – applies the update, reads back new state, persists to state backend
-- **`deleteResourceOperation`** – deletes the resource, removes the entry from state backend
-- **`readResourceOperation`** – reads current state from the cloud provider (used for drift detection)
-
-### Pending operations
-
-A resource operation throws `ResourceOperationPendingError` when it has not finished. The reconciler reads two fields from the error:
-
-| Field | Action |
-| ----- | ------ |
-| `retryAfterMs` | Wait this many milliseconds. |
-| `callbackContext` | Pass this value to the next call of the same operation. |
-
-The reconciler then calls the same operation again. Any other error fails the operation. See [Operation errors](./resource.md#operation-errors) for the complete API.
-
-The default limit is 30 calls to one operation:
-
-```ts [packages/reconciler/src/index.ts]
-{
- maxOperationAttempts: 30,
-}
-```
-
-The last pending error becomes a failure when the limit is reached.
-
-### Operation lifecycle
-
-Each operation follows the following pattern:
-
-1. Emit `started` event
-2. Execute the cloud operation (with retries)
-3. Read back the resource state
-4. Persist to state backend
-5. Emit `completed` event (or `failed` on error)
-
-State is updated after the provider operation and read-back complete.
+Lifecycle events cover create, read, update, and delete with `start`, `success`, `error`, `skip`, or `dry-run` status.

docs/internals/resource.md

@@ -173,11 +173,11 @@ new ResourceOperationPendingError(message: string, {
 | Handler result | Meaning | What the reconciler does |
 | -------------- | ------- | ------------------------ |
 | Return normally | The operation finished. | Continues the deployment. |
-| `throw new ResourceNotFoundError(message, { cause })` | `read` found no resource for the given key. | Treats the resource as absent during planning and refresh. A read after create or update fails because that operation claimed to have finished. |
+| `throw new ResourceNotFoundError(message, { cause })` | `read` or `delete` found no resource for the given key. | From `read`, treats the resource as absent during planning and drift detection; a read after create or update fails because that operation claimed to have finished. From `delete`, treats the delete as complete, since absence is its goal state. |
 | `throw new ResourceOperationPendingError(message, { retryAfterMs, callbackContext })` | The operation has not finished. | Waits for `retryAfterMs`, then calls the same handler again. It passes `callbackContext` as the handler's final argument. |
 | Throw any other error | The operation failed. | Stops the deployment. |
 
-`ResourceNotFoundError` is for `read`. A `delete` handler must catch the provider's missing-resource error and return normally.
+`ResourceNotFoundError` means the resource is absent wherever it is thrown. A `delete` handler may either catch the provider's missing-resource error and return normally, or translate it to `ResourceNotFoundError`; both count as success.
 
 `ResourceOperationPendingError` may be thrown by `create`, `read`, `update`, or `delete`. Its options are:
 
@@ -187,7 +187,7 @@ new ResourceOperationPendingError(message: string, {
 | `callbackContext` | `Readonly<Record<string, unknown>>` | no | Plain serializable data for the next attempt. |
 | `cause` | `unknown` | no | The provider error that caused this result. |
 
-The default limit is 30 attempts. Set `maxOperationAttempts` on the reconciler to change it. Reaching the limit fails the operation.
+The default limit is 30 attempts. Set `maxOperationAttempts` in the deploy, plan, or destroy options to change it. Reaching the limit fails the operation.
 
 ```ts
 read: async (key, context) => {