diff --git a/.changeset/reconciler.md b/.changeset/reconciler.md index 409f877..e304772 100644 --- a/.changeset/reconciler.md +++ b/.changeset/reconciler.md @@ -6,9 +6,6 @@ "@notation/reconciler": minor "@notation/resource": minor "@notation/state": minor -"@notation/state-sqlite": minor --- -Add the reconciler API, versioned event streams, renewable mutation leases, -SQLite state, backend-neutral dashboard state, and compiled infrastructure -graphs. +Add durable Yieldstar 0.5.0 deploy and destroy workflows, a resident Node SQLite runtime for CLI execution, versioned event streams, backend-neutral dashboard state, and compiled infrastructure graphs. diff --git a/docs/cli/dashboard.md b/docs/cli/dashboard.md index f630ad7..e51fd92 100644 --- a/docs/cli/dashboard.md +++ b/docs/cli/dashboard.md @@ -1,21 +1,13 @@ # notation dashboard ```sh -notation dashboard +notation dashboard ``` -Starts a local web dashboard for observing deployment state. +Starts a local web dashboard for observing the deployment's resource state. ```sh -notation dashboard +notation dashboard infra/api.ts ``` -The dashboard uses the same state backend as deploy and destroy. Set -`NOTATION_STATE_PATH` to select SQLite: - -```sh -NOTATION_STATE_PATH=.notation/state.db notation dashboard -``` - -The server reads through `StateBackend`, so file and SQLite state produce the same -dashboard payload. +The dashboard reads `.notation/workflows.db`, the same database used by deploy, destroy, and plan. Set `NOTATION_DATABASE_PATH` to choose another SQLite database path. diff --git a/docs/cli/deploy.md b/docs/cli/deploy.md index 940cf7e..8020225 100644 --- a/docs/cli/deploy.md +++ b/docs/cli/deploy.md @@ -4,7 +4,7 @@ notation deploy ``` -Compiles and deploys the stack to AWS. +Compiles the stack and runs a durable deploy. ```sh notation deploy infra/api.ts @@ -12,32 +12,36 @@ notation deploy infra/api.ts ## Event stream -`--json` writes versioned reconciler events to stdout as newline-delimited JSON. Build -output and diagnostics move to stderr. +`--json` writes versioned reconciler events to stdout as newline-delimited JSON. Build output, the execution ID, and diagnostics move to stderr. ```sh notation deploy infra/api.ts --json > deploy.ndjson ``` +## Durable execution + +The command prints its execution ID before starting provider work. If the process crashes, resume the same execution with that ID: + +```sh +notation deploy infra/api.ts --execution-id +``` + +Do not reuse a completed execution ID for a new deploy or for destroy. + +Retryable provider conditions and consistency reads suspend on durable SQLite timers. The CLI stays resident until the scheduler wakes the execution and the workflow completes. Provider results are replayed after their heap checkpoint, but a crash after the provider accepts a create or update and before that checkpoint repeats the call, so provider mutations must be idempotent. Reconciler event consumers must tolerate the equivalent duplicate-delivery window. + ## What happens -1. **Compile** – esbuild compiles infra and runtime modules to `dist/`. +1. **Compile** – esbuild compiles infrastructure and runtime modules to `dist/`. -2. **Build resource graph** – imports the compiled output and collects the declared resources. +2. **Build resource graph** – the CLI imports the compiled output in-process and collects declared resources. -3. **Reconcile** – the reconciler compares desired state (graph) against current state (`.notation/state.json`): - - New resources → **create** - - Changed params → **update** - - No changes → **noop** - - Orphaned resources (in state but not in graph) → **delete** +3. **Order dependencies** – dependency levels run in topological order. -4. **Topological deployment** – resources deploy in dependency order (levels). Resources at the same level deploy concurrently. +4. **Reconcile** – Notation compares desired resources with Yieldstar stores, then creates, updates, recreates, or leaves each resource unchanged. -5. **Drift detection** – enabled by default. Reads actual AWS state and compares against stored state. If drifted, Notation updates to match your definition. +5. **Detect drift** – unchanged resources are read from the provider and repaired when their remote state differs. -State is persisted to `.notation/state.json` after each operation. Set -`NOTATION_STATE_PATH` to a path ending in `.db` or `.sqlite` to use SQLite: +6. **Delete orphans** – persisted resources absent from the graph are deleted when their resource type is registered. -```sh -NOTATION_STATE_PATH=.notation/state.db notation deploy infra/api.ts -``` +State, step results, timers, queued tasks, and resource stores are persisted to `.notation/workflows.db`. Set `NOTATION_DATABASE_PATH` to choose another SQLite database path. diff --git a/docs/cli/destroy.md b/docs/cli/destroy.md index a74ff30..36e2209 100644 --- a/docs/cli/destroy.md +++ b/docs/cli/destroy.md @@ -4,7 +4,7 @@ notation destroy ``` -Removes all resources in the stack. Tears down runs in reverse dependency order, so routes are removed before APIs and Lambdas before IAM roles etc. +Compiles the application and runs a durable destroy. Resources are removed in reverse dependency order, then registered persisted orphans are removed. ```sh notation destroy infra/api.ts @@ -15,3 +15,13 @@ notation destroy infra/api.ts ```sh notation destroy infra/api.ts --json > destroy.ndjson ``` + +The command prints its execution ID. Resume a crashed destroy with the same ID so checkpointed work can be replayed: + +```sh +notation destroy infra/api.ts --execution-id +``` + +Retryable deletes suspend on durable SQLite timers. Resource state is removed only after the provider delete succeeds or reports that the resource is already absent. + +The crash-window contract described under [notation deploy](./deploy.md#durable-execution) applies to deletes as well. diff --git a/docs/internals/reconciler.md b/docs/internals/reconciler.md index 6b0d413..2d3b1ca 100644 --- a/docs/internals/reconciler.md +++ b/docs/internals/reconciler.md @@ -1,115 +1,52 @@ # Reconciler -The reconciler runs deployment operations to transition infrastructure from its current state to the state defined in the project. - -Source: `@notation/reconciler` +The reconciler expresses deployment and destruction as Yieldstar async generators. Notation owns desired-state decisions and provider lifecycle; the caller's Yieldstar runtime owns durable execution, waiting, and shared state. ## Deploy flow -```ts [packages/reconciler/src/index.ts] -const reconciler = new Reconciler({ state, registry, emit }); -await reconciler.deploy(resources, { dryRun, driftDetection }); -``` - -The reconciler walks the resource graph and, for each resource, determines an action: - -| Condition | Decision | -| --------------------------------------------- | ------------------ | -| Not in state | **create** | -| In state, params changed | **update** | -| In state, params unchanged, no drift | **noop** | -| In state, but deleted from AWS | **drift-recreate** | -| In state, AWS state differs from stored state | **drift-update** | -| In state, not in graph (orphan) | **delete** | - -The `dryRun` flag runs the full diffing pipeline without executing any operations, so you can preview what a deploy would do. - -## Topological ordering +`deploy` takes the deployment hold, walks dependency levels in order, decides an action for every resource, executes provider calls as durable steps, persists the result in a resource store, and deletes registered orphans. -Resources are deployed in dependency order using `buildResourceDepthLevels()`. This function partitions the resource graph into levels – each level contains resources whose dependencies have all been satisfied by previous levels. +| Condition | Decision | +| --- | --- | +| Not in state | **create** | +| In state, params changed | **update** | +| In state, params unchanged, no drift | **noop** | +| In state, but deleted from the provider | **drift-recreate** | +| In state, provider state differs from stored state | **drift-update** | +| In state, not in graph | **delete-orphan** | -``` -Level 0: IAM Role, CloudWatch LogGroup -Level 1: Lambda Function (depends on Role, LogGroup) -Level 2: API Gateway Integration (depends on Lambda) -Level 3: API Gateway Route (depends on Integration) -``` +Dry-run deploy performs decisions and emits lifecycle events without provider mutations or state mutations. When drift detection is enabled, it can still call provider read operations to decide whether a nominal noop has drifted. -Resources within a level deploy concurrently, so independent resources like the IAM Role and LogGroup above are provisioned in parallel. Dependent resources wait for their dependencies. +## Destroy flow -Destroy operates in reverse order with dependents getting removed before their dependencies. +`destroy` is a first-class durable operation. It takes the same deployment hold as deploy, deletes desired resources in reverse dependency order, deletes registered persisted orphans, and conditionally removes each resource store only after the provider delete succeeds or reports that the resource is already absent. -### Cycle detection +## Waiting and replay -Cycle detection is built in. If resources form a circular dependency, the build fails with: +Provider calls are stable durable steps, but provider acknowledgement and the Yieldstar heap checkpoint are not atomic. If the process crashes between them, replay repeats the call, so provider create, update, and delete operations must be idempotent. Event subscribers must likewise tolerate duplicate delivery when a crash occurs before the event checkpoint. -``` -Resource dependency cycle detected -``` +A resource operation throws `ResourceOperationPendingError` when it has not finished. The error gives the reconciler a delay and optional callback context. The runtime stores the context, waits without keeping the process busy, and calls the same operation again. See [Operation errors](./resource.md#operation-errors) for the complete API. -This catches configuration errors before any cloud operations are attempted. +Each attempt, delay, event, state read, state write, and hold change has a stable step key. A resumed execution must use the same execution ID. A new deploy or destroy must use a new execution ID. -## Drift detection +## State and the deployment hold -Drift detection is enabled by default. After confirming no local changes to a resource, the reconciler reads the resource's current state from AWS (via the resource's `read()` operation) and diffs it against stored state. +Each resource is stored under `notation/resource-state` with a deployment-scoped ID. Conditional updates and deletes compare the snapshot's UUIDv7 `instanceId` and version, so a stale execution cannot modify a deleted and recreated store. -If AWS has drifted (e.g. someone changed a Lambda timeout in the console, or an IAM policy was modified by another tool), Notation updates the resource to match the canoncial definition in the source code. +Deploy and destroy take an exclusive hold on the deployment through one `notation/deployment-hold` store per deployment. `store.take` suspends a competing execution as a durable waiter and wakes it after the holder releases. A waiter that finds the hold already taken when it inspects it emits `reconciler.hold.waiting` naming the holding execution ID, so a wait behind a crashed execution is visible instead of silent; a holder that appears only between that inspection and the `take` suspends the waiter without the event. -Properties marked as `volatile` in the schema (like `LastModified` timestamps) are excluded from drift comparison. +A failed or suspended execution keeps its hold, which is what makes resuming it safe. The hold of an execution that will never be resumed is cleared with `clearDeploymentHold` from `@notation/reconciler/durable` — the only supported way out of that state. ## Events -The reconciler emits events at each step of an operation's lifecycle. The default `createConsoleReconcilerSubscriber()` logs these to the console with formatted output. +The durable workflows emit these events: | Event | When | | ------------------------------------ | --------------------------------------------------- | | `reconciler.deploy.decision` | After deciding what action to take for a resource | | `reconciler.drift.detected` | When drift is found between stored and actual state | | `reconciler.operation.lifecycle` | When an operation starts, finishes, skips, or fails | +| `reconciler.hold.waiting` | When another execution holds the deployment | | `reconciler.orphan-deletion.skipped` | When no registered class can delete an orphan | -Lifecycle events contain the operation (`create`, `read`, `update`, or `delete`) and its -status (`start`, `success`, `error`, `skip`, or `dry-run`). Events carry the resource ID, -type, and relevant diff or error details. - -## Operations - -Each CRUD operation is implemented as an async generator with retry support: - -- **`createResourceOperation`** – creates the resource, reads back its state, persists to state backend -- **`updateResourceOperation`** – applies the update, reads back new state, persists to state backend -- **`deleteResourceOperation`** – deletes the resource, removes the entry from state backend -- **`readResourceOperation`** – reads current state from the cloud provider (used for drift detection) - -### Pending operations - -A resource operation throws `ResourceOperationPendingError` when it has not finished. The reconciler reads two fields from the error: - -| Field | Action | -| ----- | ------ | -| `retryAfterMs` | Wait this many milliseconds. | -| `callbackContext` | Pass this value to the next call of the same operation. | - -The reconciler then calls the same operation again. Any other error fails the operation. See [Operation errors](./resource.md#operation-errors) for the complete API. - -The default limit is 30 calls to one operation: - -```ts [packages/reconciler/src/index.ts] -{ - maxOperationAttempts: 30, -} -``` - -The last pending error becomes a failure when the limit is reached. - -### Operation lifecycle - -Each operation follows the following pattern: - -1. Emit `started` event -2. Execute the cloud operation (with retries) -3. Read back the resource state -4. Persist to state backend -5. Emit `completed` event (or `failed` on error) - -State is updated after the provider operation and read-back complete. +Lifecycle events cover create, read, update, and delete with `start`, `success`, `error`, `skip`, or `dry-run` status. diff --git a/docs/internals/resource.md b/docs/internals/resource.md index baae261..dc2a26f 100644 --- a/docs/internals/resource.md +++ b/docs/internals/resource.md @@ -173,11 +173,11 @@ new ResourceOperationPendingError(message: string, { | Handler result | Meaning | What the reconciler does | | -------------- | ------- | ------------------------ | | Return normally | The operation finished. | Continues the deployment. | -| `throw new ResourceNotFoundError(message, { cause })` | `read` found no resource for the given key. | Treats the resource as absent during planning and refresh. A read after create or update fails because that operation claimed to have finished. | +| `throw new ResourceNotFoundError(message, { cause })` | `read` or `delete` found no resource for the given key. | From `read`, treats the resource as absent during planning and drift detection; a read after create or update fails because that operation claimed to have finished. From `delete`, treats the delete as complete, since absence is its goal state. | | `throw new ResourceOperationPendingError(message, { retryAfterMs, callbackContext })` | The operation has not finished. | Waits for `retryAfterMs`, then calls the same handler again. It passes `callbackContext` as the handler's final argument. | | Throw any other error | The operation failed. | Stops the deployment. | -`ResourceNotFoundError` is for `read`. A `delete` handler must catch the provider's missing-resource error and return normally. +`ResourceNotFoundError` means the resource is absent wherever it is thrown. A `delete` handler may either catch the provider's missing-resource error and return normally, or translate it to `ResourceNotFoundError`; both count as success. `ResourceOperationPendingError` may be thrown by `create`, `read`, `update`, or `delete`. Its options are: @@ -187,7 +187,7 @@ new ResourceOperationPendingError(message: string, { | `callbackContext` | `Readonly>` | no | Plain serializable data for the next attempt. | | `cause` | `unknown` | no | The provider error that caused this result. | -The default limit is 30 attempts. Set `maxOperationAttempts` on the reconciler to change it. Reaching the limit fails the operation. +The default limit is 30 attempts. Set `maxOperationAttempts` in the deploy, plan, or destroy options to change it. Reaching the limit fails the operation. ```ts read: async (key, context) => { diff --git a/docs/internals/state.md b/docs/internals/state.md index 44d5df0..cb0bd8e 100644 --- a/docs/internals/state.md +++ b/docs/internals/state.md @@ -1,129 +1,15 @@ # State -Notation tracks deployed resources in a state backend. State is the bridge between what is defined and what actually exists in the cloud. +Notation CLI deploy, destroy, plan, and dashboard use Yieldstar stores in `.notation/workflows.db`. Override the database path with `NOTATION_DATABASE_PATH`. -Source: `@notation/state` - -## State file - -Default location: `.notation/state.json`. Override with the `NOTATION_STATE_PATH` environment variable. - -Each resource entry records everything needed to diff, update, or delete the resource: - -```json -{ - "my-api-lambda-getTodos": { - "rev": 3, - "id": "my-api-lambda-getTodos", - "type": "aws/lambda/LambdaFunction", - "config": { - "service": "aws/lambda", - "timeout": 5, - "memory": 64 - }, - "params": { - "FunctionName": "my-api-getTodos", - "Runtime": "nodejs18.x", - "Handler": "index.getTodos", - "MemorySize": 64, - "Timeout": 5 - }, - "output": { - "FunctionArn": "arn:aws:lambda:us-east-1:123456789:function:my-api-getTodos", - "FunctionUrl": "https://xyz.lambda-url.us-east-1.on.aws/" - }, - "lastOperation": "create", - "lastOperationAt": "2027-01-15T10:30:00.000Z" - } -} -``` - -Key fields: - -- **`id`** – unique identifier derived from the resource's position in the graph -- **`rev`** – monotonically increasing revision used for compare-and-swap writes -- **`type`** – the resource type string (e.g., `aws/lambda/LambdaFunction`) -- **`config`** – user-facing configuration values -- **`params`** – the full set of parameters sent to the cloud provider -- **`output`** – computed values returned by the provider after creation -- **`lastOperation`** – what the reconciler last did (`create`, `update`, `delete`) -- **`lastOperationAt`** – ISO timestamp of the last operation - -## Backends - -Three built-in backends: - -### `FileStateBackend` (default) - -Reads and writes JSON to disk. Uses atomic writes – writes to a temporary file first, then renames – to prevent corruption if the process is interrupted mid-write. - -```ts [packages/state/src/file.ts] -const state = new FileStateBackend(".notation/state.json"); -``` - -### `MemoryStateBackend` - -In-memory backend used for testing. Deep-clones on read and write to simulate persistence semantics (mutations to returned objects don't affect stored data). - -```ts [packages/state/src/memory.ts] -const state = new MemoryStateBackend(); -``` - -### `SqliteStateBackend` - -Stores state and leases in SQLite. Select it in the CLI by setting -`NOTATION_STATE_PATH` to a path ending in `.db` or `.sqlite`. +Each live resource is a `notation/resource-state` store scoped by deployment and resource ID. A missing store means the resource is absent. ```ts -const state = new SqliteStateBackend(".notation/state.db"); -``` - -### `StateBackend` interface - -All backends implement the same interface: - -```ts [@notation/state/src/backend.ts] -interface StateBackend { - get(id: string): Promise; - has(id: string): Promise; - update( - id: string, - patch: Partial, - expectedRev?: number, - ): Promise<{ rev: number }>; - delete(id: string, expectedRev?: number): Promise; - values(): Promise; - lease(scope: string, ttl: number): Promise; -} +const state = new DurableStateBackend(storeClient, "infra/api.ts"); ``` -Every backend provides compare-and-swap writes and renewable exclusive leases. The -reconciler holds a per-resource lease across the provider operation and state write, so -concurrent deploys cannot both perform the same create or update. It renews long-running -leases until the mutation finishes. Orphan deletion additionally holds a snapshot lease -while it decides which state records no longer appear in the desired graph. - -## How state is used - -### Deploy - -The reconciler reads state to diff against the desired resource graph: - -1. For each resource in the graph, check if it exists in state -2. If it exists, compare `params` to detect changes -3. Execute the appropriate operation (create, update, noop) -4. After each operation, update the state entry with new params and output - -### Destroy - -The reconciler reads state to find resources to delete: - -1. Load all state entries -2. Delete resources in reverse dependency order -3. Remove each entry from state after successful deletion - -### Orphan detection +The runtime assigns a UUIDv7 `instanceId` when a store is created and increments its version on update. Conditional workflow updates and deletes compare both values, preventing a stale snapshot from modifying a deleted and recreated resource. The store version is exposed unchanged as `StateNode.version`. -The reconciler checks for orphaned resources – resources that exist in state but are no longer present in the resource graph. This happens when you remove a function export or delete a `.fn.ts` file. +`DurableStateBackend` is read-only: state writes happen inside the workflow, through the store handle, so each write is stamped with the step that made it and is not repeated on replay. -Orphaned resources are deleted from AWS and removed from state. +The workflow serializes deploy and destroy through one `notation/deployment-hold` store per deployment. diff --git a/docs/manual/introduction.md b/docs/manual/introduction.md index bb629e0..41cafca 100644 --- a/docs/manual/introduction.md +++ b/docs/manual/introduction.md @@ -13,8 +13,7 @@ todoRouter.get("/todos", getTodos); Notation is a compiler, reconciler, and deployment engine. -The reconciler is also available as an embedded library. A Node.js host can construct -resources, choose a state backend, and run plan, deploy, or destroy without the CLI. +The reconciler is also available as an embedded library. A Node.js host can construct resources and compose durable reconciliation inside its own Yieldstar workflow without the CLI. The compiler runs two passes over your codebase: diff --git a/docs/manual/reconciler.md b/docs/manual/reconciler.md index 87b02f7..b9ca129 100644 --- a/docs/manual/reconciler.md +++ b/docs/manual/reconciler.md @@ -1,55 +1,43 @@ # Reconciler -Use the reconciler directly when a Node.js application needs to deploy resources without -starting the Notation CLI. - -This complete program deploys two static sites and keeps their deployment state in -SQLite: +Use `deploy` and `destroy` when a Node.js application needs durable resource lifecycle operations without starting the Notation CLI. Notation owns reconciliation intent, graph ordering, provider calls, and resource state; the application owns the outer Yieldstar workflow and runtime. ```ts -import { Reconciler, createResourceRegistry } from "@notation/reconciler"; -import { SqliteStateBackend } from "@notation/state-sqlite"; -import { StaticSite } from "./static-site"; - -const state = new SqliteStateBackend("sites.db"); - -const resources = [ - new StaticSite({ - id: "documentation", - config: { - siteDirectory: "sites/docs", - html: "

Documentation

\n", - }, - }), - new StaticSite({ - id: "status", - config: { - siteDirectory: "sites/status", - html: "

All systems operational

\n", - }, - }), -]; - -const reconciler = new Reconciler({ - state, - registry: createResourceRegistry([StaticSite]), +import { SqliteSchedulerClient, SqliteStoreClient, SqliteTaskQueueClient, SqliteTimersClient, createSqliteDb } from "@yieldstar/sqlite-runtime/node"; +import { DurableStateBackend, deploy as deployResources, destroy as destroyResources } from "@notation/reconciler/durable"; +import { workflow } from "yieldstar"; + +const database = createSqliteDb({ path: ".notation/workflows.db" }); +const schedulerClient = new SqliteSchedulerClient({ + taskQueueClient: new SqliteTaskQueueClient(database), + timersClient: new SqliteTimersClient(database), +}); +const storeClient = new SqliteStoreClient({ db: database, schedulerClient }); +const state = new DurableStateBackend(storeClient, "my-application"); + +export const deploy = workflow(async function* (step, event) { + yield* deployResources(step, { + executionId: event.executionId, + resources, + state, + }); }); -try { - await reconciler.deploy(resources); -} finally { - state.close(); -} +export const destroy = workflow(async function* (step, event) { + yield* destroyResources(step, { + executionId: event.executionId, + resources, + state, + }); +}); ``` -`StaticSite` contains the provider operations which create, read, update, and delete a -site. A real provider would call its infrastructure API instead of writing local files. +The outer workflow supplies durable step execution, timers, shared stores, waiting, and scheduling. Checkpointed provider results are replayed from the heap after a crash, retryable provider conditions suspend on a durable timer, and conditional state writes use Yieldstar store identity and version. Provider operations and event consumers are bound by the crash-window contract stated in [the reconciler internals](../internals/reconciler.md#waiting-and-replay). + +Each live resource is one Yieldstar store. Yieldstar's UUIDv7 store `instanceId` and version are authoritative for conditional update and delete; Notation exposes the version unchanged as the resource state's `version`. -Pass the complete desired set to `deploy`. A resource which remains in deployment state -but is absent from that set is deleted. The explicit registry lets the reconciler find -its delete operation. +Operations against the same deployment — the `deploymentId` the `DurableStateBackend` is constructed with — are serialized through a deployment hold naming the holding `executionId`. Resume a crashed operation with the same execution ID; use a new globally unique execution ID for every new deploy or destroy. An execution that must wait emits a `reconciler.hold.waiting` event naming the holder before it suspends, which also identifies a crashed holder that should be resumed instead. If the holder is genuinely abandoned, clear its hold with `clearDeploymentHold`. -Notation's state records what was deployed. It does not replace application data which -owns the desired configuration. +Pass the complete desired set on every deployment. Persisted resources absent from that set are deleted through the supplied resource registry. Destroy removes current resources in reverse dependency order and then removes any persisted orphans whose resource type is registered. -The runnable version is in `examples/reconciler`. +The runnable Node SQLite composition is in `examples/reconciler`. diff --git a/docs/rfcs/reconciler.md b/docs/rfcs/reconciler.md index dd33e92..a91a637 100644 --- a/docs/rfcs/reconciler.md +++ b/docs/rfcs/reconciler.md @@ -1,102 +1,28 @@ -# RFC: Reconciler +# RFC: Durable Yieldstar reconciliation **Status:** implemented -**Scope:** `@notation/state`, `@notation/reconciler` +**Scope:** `@notation/reconciler`, `@notation/core`, Yieldstar 0.5.0 -Notation evaluates an infrastructure program into resources, then reconciles those -resources against recorded state. The same engine now runs behind the CLI, the dashboard, -and direct library integrations. +Notation describes reconciliation intent and resource lifecycle operations. An outer Yieldstar workflow supplies durable execution, waiting, and state by composing `deploy` or `destroy`. -```ts -import { Reconciler } from "@notation/reconciler"; -import { SqliteStateBackend } from "@notation/state-sqlite"; +## Boundary -const state = new SqliteStateBackend(".notation/state.db"); -const reconciler = new Reconciler({ state }); +Live resource objects remain in the workflow process. They are not serialized into workflow parameters. This keeps provider clients and operation closures under Notation's lifecycle control while Yieldstar persists step results and shared state. -await reconciler.deploy(resources); -state.close(); -``` +Provider create, update, read, and delete calls are durable steps with stable resource-scoped keys. Once a result reaches the heap checkpoint, replay uses the cached result and continues at state persistence. The resulting crash-window contract is stated in [the reconciler internals](../internals/reconciler.md#waiting-and-replay). Retryable provider conditions become Yieldstar delays, allowing the process to wait without polling the provider continuously. -The reconciler boundary consists of live resource objects, a state backend, and an event -subscriber. Resource operations run in the host process. +## State lifecycle -## State +`DurableStateBackend` stores one live resource per `notation/resource-state` store. The store ID is scoped by deployment and resource ID. Store absence is resource absence. -Each state record carries a revision. Updates and deletes can require the revision which -the caller previously read: +The runtime-assigned UUIDv7 `instanceId` distinguishes a deleted store from a later store created under the same logical ID. Yieldstar's version is the concurrency token, exposed unchanged as `StateNode.version`. Workflow updates use `store.updateFrom` and deletes use `store.deleteFrom`, so both the instance and version must match the snapshot that informed the operation. -```ts -await state.update(resource.id, patch, resource.rev); -``` +## Deployment hold -A stale writer receives `RevConflict`. A missing record has revision zero, so -`expectedRev: 0` means that the record must not exist. +Each deployment has a `notation/deployment-hold` store shared by deploy and destroy. The workflow atomically claims it with `store.take`. A concurrent execution suspends as a durable waiter and is woken when the holder releases the store. The same execution can recover an acquisition across the store-commit and heap-write crash gap through Yieldstar's applied-step ledger. -The reconciler also takes a renewable per-resource lease before it reads a resource for -mutation. The lease remains held across the provider operation and state write. Two -hosts therefore cannot create or update the same resource concurrently through the same -backend. +## Node CLI runtime -Orphan deletion takes an additional snapshot lease. The snapshot remains stable while -the reconciler decides which state records no longer appear in the desired graph. +`NodeDurableRuntime` wires `WorkflowRunner`, `SqliteHeapClient`, `SqliteStoreClient`, `SqliteSchedulerClient`, and `SqliteEventLoop` against one Node SQLite database. CLI deploy and destroy run through this resident runtime and wait for a workflow result across timer and store wake-ups. -## Backends - -`@notation/state` provides file and memory backends. `@notation/state-sqlite` provides -the reference database backend. - -Every backend implements the same contract: - -```ts -interface StateBackend { - get(id: string): Promise; - has(id: string): Promise; - update( - id: string, - patch: Partial, - expectedRev?: number, - ): Promise<{ rev: number }>; - delete(id: string, expectedRev?: number): Promise; - values(): Promise; - lease(scope: string, ttl: number): Promise; -} -``` - -The dashboard reads this interface. It does not inspect a state file directly. - -## Events - -The reconciler accepts one subscriber: - -```ts -const reconciler = new Reconciler({ - state, - emit: async (event) => auditLog.write(event), -}); -``` - -`createNdjsonEventEmitter` adapts the subscriber to a versioned newline-delimited JSON -stream. The CLI uses the same adapter for `deploy --json` and `destroy --json`. - -## Package boundary - -The CLI creates resources from compiled Notation programs, then hands those live objects -to `Reconciler`. An application can construct the same resource classes directly. - -The reconciler does not serialise resource classes or execute operations in another -process. Detached execution needs manifests, resource-reference encoding, actuator -binding, and a runtime consumer. That work has its own RFC and release. - -## Acceptance - -The reconciler example is the compatibility test for this boundary. It must: - -1. Construct a resource without the CLI. -2. Plan and deploy it through `Reconciler`. -3. Close and reopen SQLite state. -4. Plan and apply an update. -5. Receive versioned events. -6. Destroy the resource and remove its state. - -The example lives in `examples/reconciler` and runs without cloud credentials. +The CLI prints a new execution ID for each operation. Re-running with `--execution-id ` resumes that operation from its durable heap after a process crash. diff --git a/examples/reconciler/README.md b/examples/reconciler/README.md index 428eea3..886cf4b 100644 --- a/examples/reconciler/README.md +++ b/examples/reconciler/README.md @@ -1,12 +1,8 @@ -# Reconciler +# Durable reconciler -This example deploys two static sites from an ordinary Node.js program. It does not -compile a Notation project or start the Notation CLI. +This example deploys two static sites from an ordinary Node.js program using Yieldstar for durable execution, state, retries, waiting, and the deployment hold. -[`src/index.ts`](./src/index.ts) is the complete program. It defines the desired -resources inline, opens a SQLite state backend, and passes the resources directly to the -reconciler. [`src/static-site.ts`](./src/static-site.ts) defines the local provider -operations used to create, read, update, and delete each site. +[`src/index.ts`](./src/index.ts) owns the outer workflow and Node SQLite runtime. It passes Yieldstar's `step` context to `deploy`, while [`src/static-site.ts`](./src/static-site.ts) contains only the desired resources and provider lifecycle operations. Run it from the repository root: @@ -14,9 +10,7 @@ Run it from the repository root: pnpm --filter reconciler-example demo ``` -The generated sites are written to `sites/`, and deployment state is stored in -`sites.db`. Change the resource configuration and run the command again to update the -sites. Remove a resource from the array and run it again to delete that site. +The generated sites are written to `sites/`, and the workflow heap, resource stores, timers, and deployment hold are stored in `sites.db`. Change the resource configuration and run the command again to update the sites. Remove a resource from the array and run it again to delete that site. Run the integration test with: diff --git a/examples/reconciler/package.json b/examples/reconciler/package.json index 55cbaa1..a634030 100644 --- a/examples/reconciler/package.json +++ b/examples/reconciler/package.json @@ -9,10 +9,11 @@ "typecheck": "tsc --noEmit" }, "dependencies": { + "@notation/core": "workspace:*", "@notation/reconciler": "workspace:*", "@notation/resource": "workspace:*", - "@notation/state-sqlite": "workspace:*", - "@notation/utils": "workspace:*" + "@notation/utils": "workspace:*", + "yieldstar": "0.5.0" }, "devDependencies": { "@types/node": "^22.13.4", diff --git a/examples/reconciler/src/index.ts b/examples/reconciler/src/index.ts index 4a85756..8898a7b 100644 --- a/examples/reconciler/src/index.ts +++ b/examples/reconciler/src/index.ts @@ -1,8 +1,14 @@ -import { Reconciler, createResourceRegistry } from "@notation/reconciler"; -import { SqliteStateBackend } from "@notation/state-sqlite"; +import { randomUUID } from "node:crypto"; +import { NodeDurableRuntime } from "@notation/core"; +import { createResourceRegistry } from "@notation/reconciler"; +import * as durable from "@notation/reconciler/durable"; +import { createWorkflowRouter, workflow } from "yieldstar"; import { StaticSite } from "./static-site"; -const state = new SqliteStateBackend("sites.db"); +const runtime = new NodeDurableRuntime({ + deploymentId: "static-sites", + databasePath: "sites.db", +}); const resources = [ new StaticSite({ @@ -21,13 +27,25 @@ const resources = [ }), ]; -const reconciler = new Reconciler({ - state, - registry: createResourceRegistry([StaticSite]), +const deploy = workflow(async function* (step, event) { + yield* durable.deploy(step, { + executionId: event.executionId, + resources, + state: runtime.state, + registry: createResourceRegistry([StaticSite]), + }); }); +// The resume handle for this run: rerunning with the same ID replays +// checkpointed work instead of repeating it. +const executionId = randomUUID(); +console.log(`Execution ID ${executionId}`); + try { - await reconciler.deploy(resources); + await runtime.run(createWorkflowRouter({ deploy }), { + workflowId: "deploy", + executionId, + }); } finally { - state.close(); + runtime.close(); } diff --git a/packages/cli/src/deploy.ts b/packages/cli/src/deploy.ts index 07fdad1..6c57116 100644 --- a/packages/cli/src/deploy.ts +++ b/packages/cli/src/deploy.ts @@ -3,12 +3,14 @@ import { createNdjsonEventEmitter, deployApp, } from "@notation/core"; +import { randomUUID } from "node:crypto"; import { compile } from "./compile"; import { defaultLogger, type Logger } from "./logger"; import { redirectStdoutToStderr } from "./stdio"; export type DeployCommandOptions = { json?: boolean; + executionId?: string; logger?: Logger; }; @@ -17,30 +19,14 @@ export async function deploy( opts: DeployCommandOptions = {}, ) { const logger = opts.logger ?? defaultLogger; - // In --json mode console output moves to stderr so stdout carries only the - // NDJSON event stream; capture the real stdout for the emitter first. const emit = opts.json ? createNdjsonEventEmitter(redirectStdoutToStderr().write) : createLoggerReconcilerSubscriber({ logger }); await compile(entryPoint, { logger }); logger.info(`Deploying ${entryPoint}`); + const executionId = opts.executionId ?? randomUUID(); + logger.info(`Execution ID ${executionId}`); - try { - await deployApp({ - entryPoint, - emit, - }); - } catch (err: any) { - if (err.name === "CredentialsProviderError") { - logger.error( - "\nAWS credentials not found.", - "\n\nEnsure you have a default profile set up in ~/.aws/credentials.", - "\n\nIf using another profile run AWS_PROFILE=otherProfile notation deploy.\n", - ); - process.exit(1); - } - logger.error(err); - process.exit(1); - } + await deployApp({ entryPoint, emit, executionId }); } diff --git a/packages/cli/src/destroy.ts b/packages/cli/src/destroy.ts index acdc5b0..177e2d7 100644 --- a/packages/cli/src/destroy.ts +++ b/packages/cli/src/destroy.ts @@ -3,12 +3,14 @@ import { createNdjsonEventEmitter, destroyApp, } from "@notation/core"; +import { randomUUID } from "node:crypto"; import { compile } from "./compile"; import { defaultLogger, type Logger } from "./logger"; import { redirectStdoutToStderr } from "./stdio"; export type DestroyCommandOptions = { json?: boolean; + executionId?: string; logger?: Logger; }; @@ -23,5 +25,8 @@ export async function destroy( await compile(entryPoint, { logger }); logger.info(`Destroying ${entryPoint}\n`); - await destroyApp({ entryPoint, emit }); + const executionId = opts.executionId ?? randomUUID(); + logger.info(`Execution ID ${executionId}`); + + await destroyApp({ entryPoint, emit, executionId }); } diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 3ef90b7..f2d7fd1 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -4,10 +4,12 @@ import { compile } from "./compile"; import { deploy } from "./deploy"; import { destroy } from "./destroy"; import { plan } from "./plan"; +import { defaultLogger } from "./logger"; +import { runWithErrorHandling } from "./run-with-error-handling"; import { visualise } from "./visualise"; import { watch } from "./watch"; import { startDashboardServer } from "@notation/dashboard"; -import { createDefaultStateBackend } from "@notation/core"; +import { NodeDurableRuntime, resolveDeploymentId } from "@notation/core"; program .command("compile") @@ -19,9 +21,13 @@ program program .command("dashboard") + .argument("", "entryPoint") .description("Start Notation Dashboard") - .action(async () => { - await startDashboardServer({ state: createDefaultStateBackend() }); + .action(async (entryPoint) => { + const runtime = new NodeDurableRuntime({ + deploymentId: resolveDeploymentId(entryPoint), + }); + await startDashboardServer({ state: runtime.state }); }); program @@ -29,8 +35,12 @@ program .argument("", "entryPoint") .description("Deploy Notation App") .option("--json", "stream reconciler events as NDJSON") + .option("--execution-id ", "resume a durable execution") .action(async (entryPoint, options) => { - await deploy(entryPoint, { json: options.json }); + await deploy(entryPoint, { + json: options.json, + executionId: options.executionId, + }); }); program @@ -38,8 +48,12 @@ program .argument("", "entryPoint") .description("Destroy Notation App") .option("--json", "stream reconciler events as NDJSON") + .option("--execution-id ", "resume a durable execution") .action(async (entryPoint, options) => { - await destroy(entryPoint, { json: options.json }); + await destroy(entryPoint, { + json: options.json, + executionId: options.executionId, + }); }); program @@ -67,4 +81,7 @@ program await watch(entryPoint); }); -program.parse(process.argv); +process.exitCode = await runWithErrorHandling( + () => program.parseAsync(process.argv), + { logger: defaultLogger, command: process.argv[2] ?? program.name() }, +); diff --git a/packages/cli/src/plan.ts b/packages/cli/src/plan.ts index ef52ec8..5880ad5 100644 --- a/packages/cli/src/plan.ts +++ b/packages/cli/src/plan.ts @@ -25,41 +25,29 @@ const decisionSymbols: Record = { export async function plan(entryPoint: string, opts: PlanCommandOptions = {}) { const logger = opts.logger ?? defaultLogger; const emit = createLoggerReconcilerSubscriber({ logger }); - try { - if (opts.json) { - let result: Plan; - const { restore } = redirectStdoutToStderr(); - try { - await compile(entryPoint, { logger }); - result = await planApp({ - entryPoint, - emit, - }); - } finally { - restore(); - } - process.stdout.write(`${JSON.stringify(result, null, 2)}\n`); - return; + if (opts.json) { + let result: Plan; + const { restore } = redirectStdoutToStderr(); + try { + await compile(entryPoint, { logger }); + result = await planApp({ + entryPoint, + emit, + }); + } finally { + restore(); } - - await compile(entryPoint, { logger }); - logger.info(`Planning ${entryPoint}\n`); - const result = await planApp({ - entryPoint, - emit, - }); - printPlanSummary(result, logger); - } catch (err: any) { - if (err.name === "CredentialsProviderError") { - logger.error( - "\nAWS credentials not found.", - "\n\nEnsure you have a default profile set up in ~/.aws/credentials.", - "\n\nIf using another profile run AWS_PROFILE=otherProfile notation plan.\n", - ); - process.exit(1); - } - throw err; + process.stdout.write(`${JSON.stringify(result, null, 2)}\n`); + return; } + + await compile(entryPoint, { logger }); + logger.info(`Planning ${entryPoint}\n`); + const result = await planApp({ + entryPoint, + emit, + }); + printPlanSummary(result, logger); } function printPlanSummary(result: Plan, logger: Logger) { diff --git a/packages/cli/src/run-with-error-handling.ts b/packages/cli/src/run-with-error-handling.ts new file mode 100644 index 0000000..552f788 --- /dev/null +++ b/packages/cli/src/run-with-error-handling.ts @@ -0,0 +1,22 @@ +import type { Logger } from "./logger"; + +export async function runWithErrorHandling( + fn: () => Promise, + opts: { logger: Logger; command: string }, +): Promise<0 | 1> { + try { + await fn(); + return 0; + } catch (error: unknown) { + if (error instanceof Error && error.name === "CredentialsProviderError") { + opts.logger.error( + "\nAWS credentials not found.", + "\n\nEnsure you have a default profile set up in ~/.aws/credentials.", + `\n\nIf using another profile run AWS_PROFILE=otherProfile notation ${opts.command}.\n`, + ); + return 1; + } + opts.logger.error(error); + return 1; + } +} diff --git a/packages/cli/src/watch.ts b/packages/cli/src/watch.ts index bc2b6b2..e87e389 100644 --- a/packages/cli/src/watch.ts +++ b/packages/cli/src/watch.ts @@ -1,4 +1,5 @@ import chokidar from "chokidar"; +import { randomUUID } from "node:crypto"; import { createLoggerReconcilerSubscriber, deployApp } from "@notation/core"; import { compile } from "./compile"; import { defaultLogger, type Logger } from "./logger"; @@ -40,8 +41,11 @@ export async function watch( isDeploying = true; + const executionId = randomUUID(); + logger.info(`Execution ID ${executionId}`); deployApp({ entryPoint, + executionId, driftDetection: false, emit: createLoggerReconcilerSubscriber({ logger }), }) diff --git a/packages/cli/test/run-with-error-handling.test.ts b/packages/cli/test/run-with-error-handling.test.ts new file mode 100644 index 0000000..343c1e1 --- /dev/null +++ b/packages/cli/test/run-with-error-handling.test.ts @@ -0,0 +1,41 @@ +import { describe, expect, it, vi } from "vitest"; +import { runWithErrorHandling } from "../src/run-with-error-handling"; + +describe("CLI error handling", () => { + it("reports credential failures with command-specific guidance", async () => { + const error = new Error("Could not load credentials"); + error.name = "CredentialsProviderError"; + const logger = { info: vi.fn(), warn: vi.fn(), error: vi.fn() }; + + const exitCode = await runWithErrorHandling( + async () => { + throw error; + }, + { logger, command: "deploy" }, + ); + + expect(exitCode).toBe(1); + expect(logger.error).toHaveBeenCalledOnce(); + expect(logger.error).toHaveBeenCalledWith( + "\nAWS credentials not found.", + "\n\nEnsure you have a default profile set up in ~/.aws/credentials.", + "\n\nIf using another profile run AWS_PROFILE=otherProfile notation deploy.\n", + ); + }); + + it("reports non-credential failures unchanged", async () => { + const error = new Error("deploy failed"); + const logger = { info: vi.fn(), warn: vi.fn(), error: vi.fn() }; + + const exitCode = await runWithErrorHandling( + async () => { + throw error; + }, + { logger, command: "deploy" }, + ); + + expect(exitCode).toBe(1); + expect(logger.error).toHaveBeenCalledOnce(); + expect(logger.error).toHaveBeenCalledWith(error); + }); +}); diff --git a/packages/core/package.json b/packages/core/package.json index 542912a..bcc145f 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -16,11 +16,15 @@ "@notation/reconciler": "workspace:*", "@notation/resource": "workspace:*", "@notation/state": "workspace:*", - "@notation/state-sqlite": "workspace:*", + "@yieldstar/core": "0.5.0", + "@yieldstar/sqlite-runtime": "0.5.0", "deep-object-diff": "^1.1.9", "js-base64": "^3.7.7", + "valibot": "^1.4.2", "lodash-es": "^4.17.21", - "pako": "^2.1.0" + "pako": "^2.1.0", + "pino": "^9.14.0", + "yieldstar": "0.5.0" }, "devDependencies": { "@types/common-tags": "^1.8.4", diff --git a/packages/core/src/provisioner/durable-runtime.ts b/packages/core/src/provisioner/durable-runtime.ts new file mode 100644 index 0000000..48385da --- /dev/null +++ b/packages/core/src/provisioner/durable-runtime.ts @@ -0,0 +1,255 @@ +import path from "node:path"; +import { setImmediate } from "node:timers/promises"; +import { isDeepStrictEqual } from "node:util"; +import { + WorkflowRunner, + type WorkflowEvent, + type WorkflowRouter, +} from "@yieldstar/core"; +import { + SqliteEventLoop, + SqliteHeapClient, + SqliteSchedulerClient, + SqliteStoreClient, + SqliteTaskQueueClient, + SqliteTimersClient, + createSqliteDb, +} from "@yieldstar/sqlite-runtime/node"; +import { + DurableStateBackend, + type DurableStep, +} from "@notation/reconciler/durable"; +import pino, { type Logger } from "pino"; +import * as v from "valibot"; +import { createWorkflowRouter, defineStore, workflow } from "yieldstar"; + +const DEFAULT_DATABASE_PATH = ".notation/workflows.db"; + +function resolveDatabasePath(): string { + return process.env.NOTATION_DATABASE_PATH ?? DEFAULT_DATABASE_PATH; +} + +export function resolveDeploymentId(entryPoint: string): string { + return path.resolve(entryPoint); +} + +export type NodeDurableRuntimeOptions = { + deploymentId: string; + databasePath?: string; + logger?: Logger; +}; + +/** + * The execution ID is required rather than defaulted: it is the handle for + * resuming a crashed execution, so the caller that starts a run must already + * hold it. Generation belongs to the outermost caller (e.g. the CLI, which + * prints the ID before any provider work). + */ +export type RunWorkflowOptions = { + workflowId: string; + executionId: string; +}; + +const executionBindingStore = defineStore( + "notation/execution-binding", + v.object({ deploymentId: v.string(), workflowId: v.string() }), +); + +type ExecutionBinding = v.InferOutput; + +/** Resident durable runtime used by Notation application commands. */ +export class NodeDurableRuntime { + readonly deploymentId: string; + readonly state: DurableStateBackend; + readonly #database: ReturnType; + readonly #eventLoop: SqliteEventLoop; + readonly #heapClient: SqliteHeapClient; + readonly #schedulerClient: SqliteSchedulerClient; + readonly #storeClient: SqliteStoreClient; + readonly #logger: Logger; + #running = false; + + constructor(opts: NodeDurableRuntimeOptions) { + this.deploymentId = opts.deploymentId; + this.#logger = opts.logger ?? pino({ level: "silent" }); + this.#database = createSqliteDb({ + path: opts.databasePath ?? resolveDatabasePath(), + }); + const taskQueueClient = new SqliteTaskQueueClient(this.#database); + this.#schedulerClient = new SqliteSchedulerClient({ + taskQueueClient, + timersClient: new SqliteTimersClient(this.#database), + }); + this.#storeClient = new SqliteStoreClient({ + db: this.#database, + schedulerClient: this.#schedulerClient, + }); + this.#heapClient = new SqliteHeapClient(this.#database); + this.#eventLoop = new SqliteEventLoop(this.#database); + this.state = new DurableStateBackend(this.#storeClient, this.deploymentId); + } + + async run( + router: WorkflowRouter, + opts: RunWorkflowOptions, + ): Promise { + if (this.#running) { + throw new Error("NodeDurableRuntime is already running a workflow"); + } + this.#running = true; + try { + const { executionId } = opts; + const runner = new WorkflowRunner({ + router, + heapClient: this.#heapClient, + storeClient: this.#storeClient, + schedulerClient: this.#schedulerClient, + logger: this.#logger, + }); + + await this.#bindExecution(executionId, opts.workflowId); + const result = await this.#driveToCompletion(runner, { + workflowId: opts.workflowId, + executionId, + params: {}, + context: new Map(), + }); + // Let the queue transaction finish before callers close the shared database. + await setImmediate(); + return result; + } finally { + this.#running = false; + } + } + + /** + * Runs one execution to completion in-process: the trigger event directly, + * then every event the queue produces for it (retries, timer wake-ups), + * polling timers between rounds. Tasks queued for other executions are + * hidden for the duration and made visible again on the way out, so this + * runner never resumes an execution it was not asked to run. + */ + async #driveToCompletion( + runner: WorkflowRunner, + event: WorkflowEvent, + ): Promise { + const deferredTaskIds: number[] = []; + try { + let outcome = await runner.run(event, this.#logger); + while (!outcome) { + const task = this.#eventLoop.taskQueue.process(); + if (!task) { + this.#eventLoop.timers.processTimers(); + await new Promise((resolve) => setTimeout(resolve, 10)); + continue; + } + if (task.event.executionId !== event.executionId) { + deferredTaskIds.push(task.taskId); + continue; + } + try { + outcome = await runner.run(task.event, this.#logger); + } finally { + this.#eventLoop.taskQueue.remove(task.taskId); + } + } + return outcome.result; + } finally { + for (const taskId of deferredTaskIds) { + this.#eventLoop.taskQueue.makeVisible(taskId); + } + } + } + + /** + * Pins an execution ID to its deployment and workflow on first use, so a + * reused ID cannot replay one workflow's cached steps inside another. + */ + async #bindExecution(executionId: string, workflowId: string): Promise { + const expected: ExecutionBinding = { + deploymentId: this.deploymentId, + workflowId, + }; + const binding = await this.#storeClient.getOrCreateStore({ + definition: executionBindingStore, + id: executionId, + initial: expected, + }); + const existing: ExecutionBinding = binding.state; + if (!isDeepStrictEqual(existing, expected)) { + throw new Error( + `Execution ${executionId} is bound to deployment ${existing.deploymentId} workflow ${existing.workflowId}, not deployment ${this.deploymentId} workflow ${workflowId}`, + ); + } + } + + close(): void { + if (this.#running) { + throw new Error( + "Cannot close NodeDurableRuntime while a workflow is running", + ); + } + this.#database.close(); + } +} + +/** + * Runs `fn` with a Node runtime for the entry point's deployment, creating one + * when the caller did not supply a runtime and closing it again afterwards. A + * supplied runtime stays open: its lifecycle belongs to the caller. + */ +export async function withRuntime( + opts: { + entryPoint: string; + runtime?: NodeDurableRuntime; + databasePath?: string; + }, + fn: (runtime: NodeDurableRuntime) => Promise, +): Promise { + if (opts.runtime && opts.databasePath) { + throw new Error( + "Pass either runtime or databasePath, not both: a runtime already owns its database", + ); + } + const runtime = + opts.runtime ?? + new NodeDurableRuntime({ + deploymentId: resolveDeploymentId(opts.entryPoint), + databasePath: opts.databasePath, + }); + try { + return await fn(runtime); + } finally { + if (!opts.runtime) runtime.close(); + } +} + +/** + * Wraps a reconciler generator as a single-workflow router and runs it to + * completion on the entry point's runtime: one command, one workflow, one + * execution. + */ +export async function runDurableWorkflow( + opts: { + entryPoint: string; + workflowId: string; + runtime?: NodeDurableRuntime; + databasePath?: string; + executionId: string; + }, + body: ( + step: DurableStep, + executionId: string, + runtime: NodeDurableRuntime, + ) => AsyncGenerator, +): Promise { + await withRuntime(opts, async (runtime) => { + const handler = workflow(async function* (step, event) { + yield* body(step, event.executionId, runtime); + }); + await runtime.run(createWorkflowRouter({ [opts.workflowId]: handler }), { + workflowId: opts.workflowId, + executionId: opts.executionId, + }); + }); +} diff --git a/packages/core/src/provisioner/index.ts b/packages/core/src/provisioner/index.ts index 89bf6e7..0b551ad 100644 --- a/packages/core/src/provisioner/index.ts +++ b/packages/core/src/provisioner/index.ts @@ -1,3 +1,3 @@ export * from "./workflows"; export * from "./resource-registry"; -export * from "./state-backend"; +export * from "./durable-runtime"; diff --git a/packages/core/src/provisioner/resource-registry.ts b/packages/core/src/provisioner/resource-registry.ts index 7d75f8f..932f3db 100644 --- a/packages/core/src/provisioner/resource-registry.ts +++ b/packages/core/src/provisioner/resource-registry.ts @@ -1,20 +1,12 @@ import { - createMissingResourceRegistryMatchWarningEvent, createResourceRegistry, createResourceRegistryFromResources, resolveResourceClass, - type MissingResourceRegistryMatchWarningEvent, type ResourceRegistry, } from "@notation/reconciler"; import type { BaseResource } from "src/orchestrator/resource"; -export { - createMissingResourceRegistryMatchWarningEvent, - createResourceRegistry, - resolveResourceClass, - type MissingResourceRegistryMatchWarningEvent, - type ResourceRegistry, -}; +export { createResourceRegistry, resolveResourceClass, type ResourceRegistry }; export function createResourceRegistryFromGraph( resources: BaseResource[], diff --git a/packages/core/src/provisioner/state-backend.ts b/packages/core/src/provisioner/state-backend.ts deleted file mode 100644 index 437e56c..0000000 --- a/packages/core/src/provisioner/state-backend.ts +++ /dev/null @@ -1,16 +0,0 @@ -import { FileStateBackend, type StateBackend } from "@notation/state"; -import { SqliteStateBackend } from "@notation/state-sqlite"; - -export const DEFAULT_STATE_PATH = "./.notation/state.json"; - -export function resolveStatePath(): string { - return process.env.NOTATION_STATE_PATH ?? DEFAULT_STATE_PATH; -} - -export function createDefaultStateBackend(): StateBackend { - const statePath = resolveStatePath(); - if (statePath.endsWith(".db") || statePath.endsWith(".sqlite")) { - return new SqliteStateBackend(statePath); - } - return new FileStateBackend(statePath); -} diff --git a/packages/core/src/provisioner/workflows/index.ts b/packages/core/src/provisioner/workflows/index.ts index 9dd1579..835d222 100644 --- a/packages/core/src/provisioner/workflows/index.ts +++ b/packages/core/src/provisioner/workflows/index.ts @@ -7,4 +7,3 @@ export { export * from "./workflow.deploy"; export * from "./workflow.destroy"; export * from "./workflow.plan"; -export * from "./workflow.refresh"; diff --git a/packages/core/src/provisioner/workflows/workflow.deploy.ts b/packages/core/src/provisioner/workflows/workflow.deploy.ts index 3eaa88e..cd102f8 100644 --- a/packages/core/src/provisioner/workflows/workflow.deploy.ts +++ b/packages/core/src/provisioner/workflows/workflow.deploy.ts @@ -1,40 +1,50 @@ +import * as durable from "@notation/reconciler/durable"; import { - Reconciler, createLoggerReconcilerSubscriber, type ReconcilerEventEmitter, type ResourceRegistry, } from "@notation/reconciler"; -import type { StateBackend } from "@notation/state"; import { getResourceGraph } from "src/orchestrator/graph"; -import { createDefaultStateBackend } from "../state-backend"; +import { runDurableWorkflow, type NodeDurableRuntime } from "../durable-runtime"; export type DeployAppOptions = { entryPoint: string; driftDetection?: boolean; dryRun?: boolean; + maxOperationAttempts?: number; registry?: ResourceRegistry; - state?: StateBackend; + runtime?: NodeDurableRuntime; + /** Required: the resume handle for this run, generated by the caller. */ + executionId: string; + databasePath?: string; emit?: ReconcilerEventEmitter; }; export async function deployApp({ entryPoint, - driftDetection = true, + // Defaulted in one place: the reconciler's drift gate treats absent as on. + driftDetection, dryRun = false, + maxOperationAttempts, registry, - state: stateBackend, + runtime, + executionId, + databasePath, emit = createLoggerReconcilerSubscriber(), }: DeployAppOptions): Promise { const graph = await getResourceGraph(entryPoint); - const state = stateBackend ?? createDefaultStateBackend(); - const reconciler = new Reconciler({ - state, - registry, - emit, - }); - - await reconciler.deploy(graph.resources, { - dryRun, - driftDetection, - }); + await runDurableWorkflow( + { entryPoint, workflowId: "deploy", runtime, databasePath, executionId }, + (step, executionId, runtime) => + durable.deploy(step, { + executionId, + resources: graph.resources, + state: runtime.state, + registry, + emit, + dryRun, + driftDetection, + maxOperationAttempts, + }), + ); } diff --git a/packages/core/src/provisioner/workflows/workflow.destroy.ts b/packages/core/src/provisioner/workflows/workflow.destroy.ts index 813239a..b7633c7 100644 --- a/packages/core/src/provisioner/workflows/workflow.destroy.ts +++ b/packages/core/src/provisioner/workflows/workflow.destroy.ts @@ -1,35 +1,43 @@ +import * as durable from "@notation/reconciler/durable"; import { - Reconciler, createLoggerReconcilerSubscriber, type ReconcilerEventEmitter, type ResourceRegistry, } from "@notation/reconciler"; -import type { StateBackend } from "@notation/state"; import { getResourceGraph } from "src/orchestrator/graph"; -import { createDefaultStateBackend } from "../state-backend"; -import { refreshState } from "./workflow.refresh"; +import { runDurableWorkflow, type NodeDurableRuntime } from "../durable-runtime"; export type DestroyAppOptions = { entryPoint: string; + maxOperationAttempts?: number; registry?: ResourceRegistry; - state?: StateBackend; + runtime?: NodeDurableRuntime; + /** Required: the resume handle for this run, generated by the caller. */ + executionId: string; + databasePath?: string; emit?: ReconcilerEventEmitter; }; export async function destroyApp({ entryPoint, + maxOperationAttempts, registry, - state: stateBackend, + runtime, + executionId, + databasePath, emit = createLoggerReconcilerSubscriber(), }: DestroyAppOptions) { - const state = stateBackend ?? createDefaultStateBackend(); - await refreshState({ entryPoint, registry, state, emit }); - const graph = await getResourceGraph(entryPoint); - const reconciler = new Reconciler({ - state, - emit, - }); - - await reconciler.destroy(graph.resources); + await runDurableWorkflow( + { entryPoint, workflowId: "destroy", runtime, databasePath, executionId }, + (step, executionId, runtime) => + durable.destroy(step, { + executionId, + resources: graph.resources, + state: runtime.state, + registry, + emit, + maxOperationAttempts, + }), + ); } diff --git a/packages/core/src/provisioner/workflows/workflow.plan.ts b/packages/core/src/provisioner/workflows/workflow.plan.ts index 2e51cb8..f0ceb4e 100644 --- a/packages/core/src/provisioner/workflows/workflow.plan.ts +++ b/packages/core/src/provisioner/workflows/workflow.plan.ts @@ -1,38 +1,42 @@ import { - Reconciler, createLoggerReconcilerSubscriber, + createPlan, type Plan, type ReconcilerEventEmitter, - type ResourceRegistry, } from "@notation/reconciler"; -import type { StateBackend } from "@notation/state"; import { getResourceGraph } from "src/orchestrator/graph"; -import { createDefaultStateBackend } from "../state-backend"; +import { withRuntime, type NodeDurableRuntime } from "../durable-runtime"; export type { Plan, PlanNode, PlanDecision } from "@notation/reconciler"; export type PlanAppOptions = { entryPoint: string; driftDetection?: boolean; - registry?: ResourceRegistry; - state?: StateBackend; + maxOperationAttempts?: number; + runtime?: NodeDurableRuntime; + databasePath?: string; emit?: ReconcilerEventEmitter; }; export async function planApp({ entryPoint, - driftDetection = true, - registry, - state: stateBackend, + // Defaulted in one place: the reconciler's drift gate treats absent as on. + driftDetection, + maxOperationAttempts, + runtime: suppliedRuntime, + databasePath, emit = createLoggerReconcilerSubscriber(), }: PlanAppOptions): Promise { const graph = await getResourceGraph(entryPoint); - const state = stateBackend ?? createDefaultStateBackend(); - const reconciler = new Reconciler({ - state, - registry, - emit, - }); - - return reconciler.plan(graph.resources, { driftDetection }); + return withRuntime( + { entryPoint, runtime: suppliedRuntime, databasePath }, + (runtime) => + createPlan({ + resources: graph.resources, + state: runtime.state, + driftDetection, + emit, + maxOperationAttempts, + }), + ); } diff --git a/packages/core/src/provisioner/workflows/workflow.refresh.ts b/packages/core/src/provisioner/workflows/workflow.refresh.ts deleted file mode 100644 index b463a87..0000000 --- a/packages/core/src/provisioner/workflows/workflow.refresh.ts +++ /dev/null @@ -1,39 +0,0 @@ -import { - Reconciler, - createLoggerReconcilerSubscriber, - type ReconcilerEventEmitter, - type ResourceRegistry, -} from "@notation/reconciler"; -import type { StateBackend } from "@notation/state"; -import { getResourceGraph } from "src/orchestrator/graph"; -import { createDefaultStateBackend } from "../state-backend"; - -/** - * @description Destroy resources that are in state but not in the orchestration graph - */ -export type RefreshStateOptions = { - entryPoint: string; - dryRun?: boolean; - registry?: ResourceRegistry; - state?: StateBackend; - emit?: ReconcilerEventEmitter; -}; - -export async function refreshState({ - entryPoint, - dryRun = false, - registry, - state: stateBackend, - emit = createLoggerReconcilerSubscriber(), -}: RefreshStateOptions): Promise { - const graph = await getResourceGraph(entryPoint); - const state = stateBackend ?? createDefaultStateBackend(); - - const reconciler = new Reconciler({ - state, - registry, - emit, - }); - - await reconciler.refresh(graph.resources, { dryRun }); -} diff --git a/packages/core/test/provisioner/durable-runtime.test.ts b/packages/core/test/provisioner/durable-runtime.test.ts new file mode 100644 index 0000000..1c0ad39 --- /dev/null +++ b/packages/core/test/provisioner/durable-runtime.test.ts @@ -0,0 +1,164 @@ +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import * as durable from "@notation/reconciler/durable"; +import { + ResourceOperationPendingError, + resource, +} from "@notation/resource"; +import { + SqliteEventLoop, + SqliteTaskQueueClient, + createSqliteDb, +} from "@yieldstar/sqlite-runtime/node"; +import { RetryableError, createWorkflowRouter, workflow } from "yieldstar"; +import { describe, expect, it } from "vitest"; +import { + NodeDurableRuntime, + resolveDeploymentId, +} from "src/provisioner/durable-runtime"; + +describe("NodeDurableRuntime", () => { + it("stays resident across a provider delay and resumes from the SQLite event loop", async () => { + const directory = await mkdtemp(path.join(tmpdir(), "notation-runtime-")); + const runtime = new NodeDurableRuntime({ + deploymentId: "resident-wait", + databasePath: path.join(directory, "workflows.db"), + }); + let attempts = 0; + const PendingResource = resource({ type: "test/runtime/pending" }) + .defineSchema({}) + .defineOperations({ + create: async () => { + attempts += 1; + if (attempts === 1) { + throw new ResourceOperationPendingError("provider is not ready", { + retryAfterMs: 10, + }); + } + }, + delete: async () => undefined, + }); + const resources = [new PendingResource({ id: "pending" })]; + const deploy = workflow(async function* (step, event) { + yield* durable.deploy(step, { + executionId: event.executionId, + resources, + state: runtime.state, + driftDetection: false, + maxOperationAttempts: 3, + }); + }); + + try { + await runtime.run(createWorkflowRouter({ deploy }), { + workflowId: "deploy", + executionId: "resident-execution", + }); + expect(attempts).toBe(2); + await expect(runtime.state.get("pending")).resolves.toMatchObject({ + lastOperation: "create", + }); + } finally { + runtime.close(); + await rm(directory, { recursive: true, force: true }); + } + }, 5_000); + + it("binds an execution ID to its deployment and workflow", async () => { + const directory = await mkdtemp(path.join(tmpdir(), "notation-binding-")); + const databasePath = path.join(directory, "workflows.db"); + const completed = workflow(async function* () {}); + const router = createWorkflowRouter({ + deploy: completed, + destroy: completed, + }); + const first = new NodeDurableRuntime({ + deploymentId: "first-deployment", + databasePath, + }); + + try { + await first.run(router, { + workflowId: "deploy", + executionId: "bound-execution", + }); + await expect( + first.run(router, { + workflowId: "destroy", + executionId: "bound-execution", + }), + ).rejects.toThrow("bound to deployment first-deployment workflow deploy"); + } finally { + first.close(); + } + + const second = new NodeDurableRuntime({ + deploymentId: "second-deployment", + databasePath, + }); + try { + await expect( + second.run(router, { + workflowId: "deploy", + executionId: "bound-execution", + }), + ).rejects.toThrow("bound to deployment first-deployment workflow deploy"); + } finally { + second.close(); + await rm(directory, { recursive: true, force: true }); + } + }); + + it("does not acknowledge queued events from another execution", async () => { + const directory = await mkdtemp(path.join(tmpdir(), "notation-queue-")); + const databasePath = path.join(directory, "workflows.db"); + const database = createSqliteDb({ path: databasePath }); + new SqliteEventLoop(database); + new SqliteTaskQueueClient(database).add({ + workflowId: "deploy", + executionId: "unrelated-execution", + params: {}, + context: new Map(), + }); + database.close(); + + let attempts = 0; + const delayed = workflow(async function* (step) { + yield* step.run("delay", async () => { + attempts += 1; + if (attempts === 1) { + throw new RetryableError("not ready", { + maxAttempts: 2, + retryInterval: 10, + }); + } + }); + }); + const runtime = new NodeDurableRuntime({ + deploymentId: "queue-test", + databasePath, + }); + try { + await runtime.run(createWorkflowRouter({ deploy: delayed }), { + workflowId: "deploy", + executionId: "current-execution", + }); + } finally { + runtime.close(); + } + + const reopened = createSqliteDb({ path: databasePath }); + const queued = new SqliteEventLoop(reopened).taskQueue.process(); + expect(queued?.event.executionId).toBe("unrelated-execution"); + reopened.close(); + await rm(directory, { recursive: true, force: true }); + }, 5_000); + + it("canonicalises equivalent entry-point spellings", () => { + const absolute = path.resolve("infra/api.ts"); + expect(resolveDeploymentId("infra/api.ts")).toBe(absolute); + expect(resolveDeploymentId("./infra/api.ts")).toBe(absolute); + expect(resolveDeploymentId(absolute)).toBe(absolute); + }); +}); diff --git a/packages/core/test/provisioner/operation.create.test.ts b/packages/core/test/provisioner/operation.create.test.ts index 2e4f506..04f4a18 100644 --- a/packages/core/test/provisioner/operation.create.test.ts +++ b/packages/core/test/provisioner/operation.create.test.ts @@ -3,8 +3,9 @@ import { createResourceOperation, createStepRunner, runOperation, + toEmitStep, + type PersistedResourceState, } from "@notation/reconciler"; -import { MemoryStateBackend } from "@notation/state"; import { TestResourceSchema, testResourceConfig, @@ -14,7 +15,7 @@ import { describe("resource creation", () => { it("passes computed input to resource.create", async () => { - const stateBackend = new MemoryStateBackend(); + let persisted: PersistedResourceState | undefined; const readResult = { ...testResourceOutput, volatileComputed: "123" }; const createMock = vi.fn(async () => ({ primaryKey: "" })); const readMock = vi.fn(async () => readResult); @@ -34,8 +35,11 @@ describe("resource creation", () => { await runOperation( createResourceOperation(step, { resource: testResource, - state: stateBackend, - expectedRev: 0, + resourceParams: await testResource.getParams(), + persist: async function* (next) { + persisted = next; + }, + emit: toEmitStep(), }), ); @@ -43,7 +47,7 @@ describe("resource creation", () => { const persistedOutput = testResource.toState(readResult); expect(createMock.mock.calls[0]).toEqual([params, undefined]); - await expect(stateBackend.get(testResource.id)).resolves.toMatchObject({ + expect(persisted).toMatchObject({ id: testResource.id, output: persistedOutput, lastOperation: "create", diff --git a/packages/core/test/provisioner/resource-registry.test.ts b/packages/core/test/provisioner/resource-registry.test.ts index e01102c..876823c 100644 --- a/packages/core/test/provisioner/resource-registry.test.ts +++ b/packages/core/test/provisioner/resource-registry.test.ts @@ -1,7 +1,6 @@ import { describe, expect, it } from "vitest"; import { resource } from "src/orchestrator/resource"; import { - createMissingResourceRegistryMatchWarningEvent, createResourceRegistry, resolveResourceClass, } from "src/provisioner/resource-registry"; @@ -21,21 +20,4 @@ describe("provisioner resource registry", () => { resolveResourceClass(registry, "test/service/unknown"), ).toBeUndefined(); }); - - it("creates a structured warning event for orphan skips", () => { - expect( - createMissingResourceRegistryMatchWarningEvent({ - workflow: "deploy", - resourceId: "orphan-id", - resourceType: "test/service/unknown", - }), - ).toEqual({ - level: "warn", - event: "reconciler.orphan-deletion.skipped", - reason: "resource-type-not-registered", - workflow: "deploy", - resourceId: "orphan-id", - resourceType: "test/service/unknown", - }); - }); }); diff --git a/packages/core/test/provisioner/state-backend.test.ts b/packages/core/test/provisioner/state-backend.test.ts deleted file mode 100644 index 5ac04a4..0000000 --- a/packages/core/test/provisioner/state-backend.test.ts +++ /dev/null @@ -1,59 +0,0 @@ -import { mkdtempSync, rmSync } from "node:fs"; -import { tmpdir } from "node:os"; -import path from "node:path"; -import { afterEach, beforeEach, describe, expect, it } from "vitest"; -import { FileStateBackend } from "@notation/state"; -import { SqliteStateBackend } from "@notation/state-sqlite"; -import { createDefaultStateBackend } from "src/provisioner/state-backend"; - -describe("createDefaultStateBackend", () => { - let directory: string; - const originalStatePath = process.env.NOTATION_STATE_PATH; - - beforeEach(() => { - directory = mkdtempSync(path.join(tmpdir(), "notation-state-")); - }); - - afterEach(() => { - if (originalStatePath === undefined) { - delete process.env.NOTATION_STATE_PATH; - } else { - process.env.NOTATION_STATE_PATH = originalStatePath; - } - rmSync(directory, { recursive: true, force: true }); - }); - - it("uses the file backend for the default JSON path", () => { - delete process.env.NOTATION_STATE_PATH; - - expect(createDefaultStateBackend()).toBeInstanceOf(FileStateBackend); - }); - - it("uses the sqlite backend for .db paths", () => { - process.env.NOTATION_STATE_PATH = path.join(directory, "state.db"); - - const backend = createDefaultStateBackend(); - expect(backend).toBeInstanceOf(SqliteStateBackend); - (backend as SqliteStateBackend).close(); - }); - - it("uses the sqlite backend for .sqlite paths", () => { - process.env.NOTATION_STATE_PATH = path.join(directory, "state.sqlite"); - - const backend = createDefaultStateBackend(); - expect(backend).toBeInstanceOf(SqliteStateBackend); - (backend as SqliteStateBackend).close(); - }); - - it("creates missing parent directories for sqlite paths", () => { - process.env.NOTATION_STATE_PATH = path.join( - directory, - ".notation", - "state.db", - ); - - const backend = createDefaultStateBackend(); - expect(backend).toBeInstanceOf(SqliteStateBackend); - (backend as SqliteStateBackend).close(); - }); -}); diff --git a/packages/dashboard/server/server.test.ts b/packages/dashboard/server/server.test.ts index e9cf2b3..8cfb300 100644 --- a/packages/dashboard/server/server.test.ts +++ b/packages/dashboard/server/server.test.ts @@ -4,11 +4,9 @@ import { readStateSnapshot } from "./server"; describe("dashboard state", () => { it("reads state through the backend contract", async () => { - const state = new MemoryStateBackend(); - await state.update( - "service", - 0, - { + const state = new MemoryStateBackend({ + service: { + version: 1, id: "service", type: "test/service/main", config: {}, @@ -17,12 +15,12 @@ describe("dashboard state", () => { lastOperation: "create", lastOperationAt: "2026-07-18T00:00:00.000Z", }, - ); + }); await expect(readStateSnapshot(state)).resolves.toMatchObject({ service: { id: "service", - rev: 1, + version: 1, output: { ready: true }, }, }); diff --git a/packages/dashboard/server/server.ts b/packages/dashboard/server/server.ts index bcc5cf7..8f0030b 100644 --- a/packages/dashboard/server/server.ts +++ b/packages/dashboard/server/server.ts @@ -6,8 +6,11 @@ import { fileURLToPath } from "node:url"; const serverDirectory = dirname(fileURLToPath(import.meta.url)); +/** The dashboard only reads state, so any backend that can list it will do. */ +export type DashboardState = Pick; + export type DashboardServerOptions = { - state: StateBackend; + state: DashboardState; pollInterval?: number; }; @@ -16,7 +19,7 @@ export type StartDashboardServerOptions = DashboardServerOptions & { }; export async function readStateSnapshot( - state: StateBackend, + state: DashboardState, ): Promise> { const nodes = await state.values(); return Object.fromEntries(nodes.map((node) => [node.id, node])); diff --git a/packages/reconciler/package.json b/packages/reconciler/package.json index 39ed2f0..c36012c 100644 --- a/packages/reconciler/package.json +++ b/packages/reconciler/package.json @@ -2,6 +2,16 @@ "type": "module", "name": "@notation/reconciler", "version": "0.12.0", + "exports": { + ".": { + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + }, + "./durable": { + "types": "./dist/durable/index.d.ts", + "default": "./dist/durable/index.js" + } + }, "main": "./dist/index.js", "types": "./dist/index.d.ts", "files": [ @@ -9,12 +19,19 @@ ], "scripts": { "build": "tsup --clean", + "typecheck": "tsc --noEmit", "dev": "tsup --watch" }, "dependencies": { "@notation/resource": "workspace:*", "@notation/state": "workspace:*", + "@yieldstar/core": "0.5.0", "deep-object-diff": "^1.1.9", - "yieldstar": "^0.4.6" + "valibot": "^1.4.2", + "yieldstar": "0.5.0" + }, + "devDependencies": { + "@yieldstar/sqlite-runtime": "0.5.0", + "pino": "^9.9.0" } } diff --git a/packages/reconciler/src/durable/deploy.ts b/packages/reconciler/src/durable/deploy.ts new file mode 100644 index 0000000..fb76450 --- /dev/null +++ b/packages/reconciler/src/durable/deploy.ts @@ -0,0 +1,30 @@ +import { buildResourceDepthLevels } from "../dependency-graph"; +import { withDeploymentHold } from "./deployment-hold"; +import { reconcileResource, sweepOrphans } from "./reconcile"; +import { scopeStep } from "./step"; +import type { DurableDeployOptions } from "./types"; +import type { DurableStep } from "./yieldstar"; + +export async function* deploy( + step: DurableStep, + opts: DurableDeployOptions, +): AsyncGenerator { + yield* withDeploymentHold(step, opts, async function* () { + // Reconcile in dependency order, so a resource only runs once its + // dependencies have converged. + for (const level of buildResourceDepthLevels(opts.resources)) { + for (const resource of level) { + yield* reconcileResource( + scopeStep( + step, + `notation:deploy:${encodeURIComponent(resource.id)}`, + ), + resource, + opts, + ); + } + } + + yield* sweepOrphans(step, opts, "deploy"); + }); +} diff --git a/packages/reconciler/src/durable/deployment-hold.ts b/packages/reconciler/src/durable/deployment-hold.ts new file mode 100644 index 0000000..770f6e4 --- /dev/null +++ b/packages/reconciler/src/durable/deployment-hold.ts @@ -0,0 +1,130 @@ +/** + * The deployment hold: an exclusive claim on a deployment for the length of a + * workflow execution. + */ +import type { ReconcilerEventEmitter } from "../events"; +import type { DurableStateBackend } from "./state-backend"; +import { durableEmitter, scopeStep } from "./step"; +import { deploymentHoldStore, type DeploymentHoldState } from "./stores"; +import type { DurableStep, StoreClient, WorkflowStore } from "./yieldstar"; + +type DeploymentHoldOptions = { + /** Deployment identity comes from the state backend. */ + state: Pick; + executionId: string; + emit?: ReconcilerEventEmitter; +}; + +/** + * Prevents concurrent executions from mutating the same deployment. Names + * the holder so an operator can resume it after a crash. + */ +async function* acquireDeploymentHold( + step: DurableStep, + opts: DeploymentHoldOptions, +): AsyncGenerator, any> { + const hold = yield* step.store(deploymentHoldStore, { + id: opts.state.deploymentId, + initial: { holder: null }, + }); + + const snapshot = yield* hold.get("notation:hold:inspect"); + const holder = snapshot.state.holder; + if (holder !== null && holder !== opts.executionId) { + yield* durableEmitter( + scopeStep(step, "notation:hold"), + opts.emit, + )({ + level: "warn", + event: "reconciler.hold.waiting", + deploymentId: opts.state.deploymentId, + executionId: opts.executionId, + holderExecutionId: holder, + }); + } + + yield* hold.take( + "notation:hold:acquire", + (state) => state.holder === null || state.holder === opts.executionId, + (draft) => { + draft.holder = opts.executionId; + }, + ); + + return hold; +} + +function releaseDeploymentHold( + hold: WorkflowStore, + executionId: string, +) { + return hold.update("notation:hold:release", (draft) => { + if (draft.holder === executionId) draft.holder = null; + }); +} + +/** + * Runs `body` while holding the deployment, releasing the hold only once + * `body` has completed. A failed or suspended execution keeps its hold: a + * resumed execution replays `take` from the step cache without re-acquiring + * anything, so it must still be the holder. An execution that will never be + * resumed holds its deployment until an operator calls + * `clearDeploymentHold`. + */ +export async function* withDeploymentHold( + step: DurableStep, + opts: DeploymentHoldOptions, + body: () => AsyncGenerator, +): AsyncGenerator { + const hold = yield* acquireDeploymentHold(step, opts); + yield* body(); + yield* releaseDeploymentHold(hold, opts.executionId); +} + +export type DeploymentHoldClearance = + | { cleared: true; previousHolder: string } + | { cleared: false; holder: string | null }; + +/** + * Clears the hold of an execution that will not be resumed, so later + * deployments are not blocked behind it. + * + * The write is conditional on `fromExecutionId` still being the named holder, + * so it cannot clear a hold that has since moved to another execution. + * Confirm the holder is genuinely dead first: clearing a live execution's + * hold permits a concurrent mutation of the same deployment. + * + * Throws if the deployment has no hold store, i.e. has never been deployed. + */ +export async function clearDeploymentHold(params: { + storeClient: StoreClient; + deploymentId: string; + fromExecutionId: string; +}): Promise { + const { storeClient, deploymentId, fromExecutionId } = params; + const read = () => + storeClient.getStore({ + definition: deploymentHoldStore, + id: deploymentId, + }); + + const snapshot = await read(); + if (snapshot.state.holder !== fromExecutionId) { + return { cleared: false, holder: snapshot.state.holder }; + } + + const result = await storeClient.updateStoreFrom({ + definition: deploymentHoldStore, + id: deploymentId, + snapshot, + updater: (draft) => { + draft.holder = null; + }, + }); + + if (!result.updated) { + return { cleared: false, holder: (await read()).state.holder }; + } + + return { cleared: true, previousHolder: fromExecutionId }; +} diff --git a/packages/reconciler/src/durable/destroy.ts b/packages/reconciler/src/durable/destroy.ts new file mode 100644 index 0000000..91df953 --- /dev/null +++ b/packages/reconciler/src/durable/destroy.ts @@ -0,0 +1,32 @@ +import { buildResourceDepthLevels } from "../dependency-graph"; +import { withDeploymentHold } from "./deployment-hold"; +import { deleteResource, sweepOrphans } from "./reconcile"; +import { scopeStep } from "./step"; +import type { DurableWorkflowOptions } from "./types"; +import type { DurableStep } from "./yieldstar"; + +/** Durably destroys persisted resources in reverse dependency order. */ +export async function* destroy( + step: DurableStep, + opts: DurableWorkflowOptions, +): AsyncGenerator { + yield* withDeploymentHold(step, opts, async function* () { + // Delete in reverse dependency order, so dependents are gone before the + // resources they depend on. + const levels = buildResourceDepthLevels(opts.resources); + for (let index = levels.length - 1; index >= 0; index -= 1) { + for (const resource of levels[index]!) { + yield* deleteResource( + scopeStep( + step, + `notation:destroy:${encodeURIComponent(resource.id)}`, + ), + resource, + opts, + ); + } + } + + yield* sweepOrphans(step, opts, "destroy"); + }); +} diff --git a/packages/reconciler/src/durable/index.ts b/packages/reconciler/src/durable/index.ts new file mode 100644 index 0000000..cd46d8a --- /dev/null +++ b/packages/reconciler/src/durable/index.ts @@ -0,0 +1,46 @@ +/** + * The durable reconciler driver: `deploy` and `destroy` as generators that a + * Yieldstar workflow composes, and the store-backed state they run against. + * + * Step keys are persisted: a resumed execution matches its cached work by + * key, so changing one re-executes the work behind it. The shapes in use are: + * + * notation:deploy::* per-resource reconciliation steps + * notation:destroy::* per-resource deletion steps + * notation:orphans:list the orphan sweep's one read of persisted state + * notation:orphans::* orphan sweep, per persisted record + * *:remote:attempt: one provider call attempt + * *:remote:retry-delay: the wait between two attempts + * *:emit:[::] event delivery checkpoint + * notation:hold:* deployment hold: inspect/acquire/release + * state:persist: conditional write of a resource record + * state:delete: conditional removal of one + * + * An inside a scope is URI-encoded, so the `:` delimiter is unambiguous. + * The state: keys are store-handle keys and so are not scope-prefixed: a + * store outlives the scope that opened it, which is why they carry the + * resource id themselves. + * + * Store names are persisted identifiers too, and carry the "notation/" + * prefix so they cannot collide with an application's stores on a shared + * store client. This driver owns: + * + * notation/resource-state one record per live resource + * notation/deployment-hold one hold per deployment + * + * @notation/core's durable runtime persists one more name under the same + * prefix, notation/execution-binding (see its durable-runtime module). + */ +export { deploy } from "./deploy"; +export { destroy } from "./destroy"; +export { + clearDeploymentHold, + type DeploymentHoldClearance, +} from "./deployment-hold"; +export { DurableStateBackend } from "./state-backend"; +export { deploymentHoldStore, resourceStateStore } from "./stores"; +export { + type DurableDeployOptions, + type DurableWorkflowOptions, +} from "./types"; +export type { DurableStep } from "./yieldstar"; diff --git a/packages/reconciler/src/durable/reconcile.ts b/packages/reconciler/src/durable/reconcile.ts new file mode 100644 index 0000000..62afa70 --- /dev/null +++ b/packages/reconciler/src/durable/reconcile.ts @@ -0,0 +1,281 @@ +/** + * Per-resource reconciliation: converging, deleting, and sweeping single + * resources, each built on one read of the resource's persisted record and + * writes conditional on that read. + */ +import type { BaseResource, ResourceType } from "@notation/resource"; +import { VersionConflict, type StateNode } from "@notation/state"; +import { + createResourceRegistryFromResources, + resolveResourceClass, +} from "../resource-registry"; +import { + applyDriftDetection, + createResourceOperation, + deleteResourceOperation, + updateResourceOperation, + type PersistState, + type RemoveState, +} from "../operations"; +import { decideAction } from "../plan"; +import { durableEmitter, scopeStep, type DurableStepRunner } from "./step"; +import { + resourceStateStore, + toStateNode, + type ResourceSnapshot, +} from "./stores"; +import type { DurableDeployOptions, DurableWorkflowOptions } from "./types"; +import type { DurableStep } from "./yieldstar"; + +/** + * A read of a resource's persisted record, together with the writes that are + * conditional on that exact read. `remove` exists only alongside a `node`: + * a record that was never read cannot be removed safely. + */ +type ResourceStateSession = + | { node: undefined; persist: PersistState; remove?: never } + | { node: StateNode; persist: PersistState; remove: RemoveState }; + +/** + * Reconciles one resource: load the persisted record, decide, read the + * remote when the decision needs it, announce the decision, then act. `step` + * must already be scoped to the resource. + */ +export async function* reconcileResource( + step: DurableStepRunner, + resource: BaseResource, + opts: DurableDeployOptions, +): AsyncGenerator { + // Resolved once and then carried: deriveParams is user code and need not be + // deterministic, so an operation resolving them again could persist params + // other than the ones the decision was taken against. The step also pins + // the answer across a replay. + const params = yield* step.run("params", () => resource.getParams()); + + const emit = durableEmitter(step, opts.emit); + const session = yield* openStateSession(step, opts, resource); + if (session.node) resource.setOutput(session.node.output); + + let action = decideAction({ resource, stateNode: session.node, params }); + + // Its own scope: when the gate fires, the operation that follows the drift + // read reads the remote again, and the two reads must not share step keys. + const driftStep = step.scope("drift-read"); + action = yield* applyDriftDetection(driftStep, { + action, + driftDetection: opts.driftDetection, + resource, + resourceParams: params, + persistedOutput: session.node?.output, + // No dryRun: a dry run suppresses mutations, not reads, and reading is + // how a dry run reports drift at all. + emit: durableEmitter(driftStep, opts.emit), + maxOperationAttempts: opts.maxOperationAttempts, + }); + + if (action.decision === "drift-update") { + yield* emit({ + level: "info", + event: "reconciler.drift.detected", + resourceId: resource.id, + resourceType: resource.type, + diff: action.patch, + }); + } + + yield* emit({ + level: "info", + event: "reconciler.deploy.decision", + resourceId: resource.id, + resourceType: resource.type, + decision: action.decision, + }); + + const shared = { + resource, + resourceParams: params, + persistedOutput: session.node?.output, + dryRun: opts.dryRun, + emit, + maxOperationAttempts: opts.maxOperationAttempts, + }; + + switch (action.decision) { + case "create": + case "drift-recreate": + yield* createResourceOperation(step, { + ...shared, + persist: session.persist, + }); + return; + case "update": + case "drift-update": + yield* updateResourceOperation(step, { + ...shared, + patch: action.patch, + persist: session.persist, + }); + return; + case "noop": + return; + } +} + +/** + * Deletes one resource. A resource with no persisted record was never created + * — or has already been deleted — and is skipped, which is also what makes + * the sweep of a partly-deleted deployment idempotent. `step` must already be + * scoped to the resource. + */ +export async function* deleteResource( + step: DurableStepRunner, + resource: BaseResource, + opts: DurableWorkflowOptions, +): AsyncGenerator { + const session = yield* openStateSession(step, opts, resource); + if (!session.node) return; + resource.setOutput(session.node.output); + + yield* deleteResourceOperation(step, { + resource, + dryRun: opts.dryRun, + emit: durableEmitter(step, opts.emit), + maxOperationAttempts: opts.maxOperationAttempts, + remove: session.remove, + }); +} + +/** + * Deletes persisted resources that are no longer in the desired set. A state + * node whose type has no registry entry is left in place and surfaced as a + * warning, because deleting it would need a resource class we cannot resolve. + */ +export async function* sweepOrphans( + step: DurableStep, + opts: DurableWorkflowOptions, + workflow: "deploy" | "destroy", +): AsyncGenerator { + const scope = scopeStep(step, "notation:orphans"); + const resourceById = new Map( + opts.resources.map((resource) => [resource.id, resource]), + ); + const persisted = yield* scope.run("list", () => opts.state.values()); + const registry = + opts.registry ?? createResourceRegistryFromResources(opts.resources); + + for (const node of persisted) { + if (resourceById.has(node.id)) continue; + const nodeScope = scope.scope(encodeURIComponent(node.id)); + + const Resource = resolveResourceClass(registry, node.type as ResourceType); + if (!Resource) { + const emit = durableEmitter(nodeScope, opts.emit); + yield* emit({ + level: "warn", + event: "reconciler.orphan-deletion.skipped", + reason: "resource-type-not-registered", + workflow, + resourceId: node.id, + resourceType: node.type as ResourceType, + }); + continue; + } + + const resource = new Resource({ id: node.id, config: node.config }); + resource.setOutput(node.output); + yield* deleteResource(nodeScope, resource, opts); + } +} + +/** + * Reads the persisted record once and binds the writes conditional on it. + * + * The snapshot is the precondition: it names the exact store instance and + * version the record was read at, so a write made against it cannot land on a + * record another writer has moved on. + */ +async function* openStateSession( + step: DurableStepRunner, + opts: DurableWorkflowOptions, + resource: BaseResource, +): AsyncGenerator { + const snapshot = yield* step.run("persisted-record", () => + opts.state.snapshot(resource.id), + ); + const persist = persistResourceState(step, opts, resource, snapshot); + if (!snapshot) return { node: undefined, persist }; + + return { + node: toStateNode(snapshot), + persist, + remove: removeResourceState(step, opts, resource, snapshot), + }; +} + +/** + * State writes go through the workflow store, never through the state backend: + * the store stamps the write with the step that made it, so the applied-step + * ledger and the state change commit together. Replaying then returns the + * recorded result instead of retrying a compare-and-set that would now fail. + */ +function persistResourceState( + step: DurableStepRunner, + opts: DurableWorkflowOptions, + resource: BaseResource, + snapshot: ResourceSnapshot | undefined, +): PersistState { + return async function* (next) { + if (!snapshot) { + // Create-if-absent. A racing writer would win here and this record would + // be silently adopted rather than written, which is safe only because a + // deployment is held exclusively for the length of the workflow. + yield* step.store(resourceStateStore, { + id: opts.state.storeId(resource.id), + initial: next, + }); + return; + } + + const store = yield* step.store(resourceStateStore, { + id: opts.state.storeId(resource.id), + }); + const result = yield* store.updateFrom( + `state:persist:${resource.id}`, + snapshot, + () => next, + ); + if (!result.updated) { + throw new VersionConflict( + resource.id, + snapshot.version, + result.actualVersion, + ); + } + }; +} + +function removeResourceState( + step: DurableStepRunner, + opts: DurableWorkflowOptions, + resource: BaseResource, + snapshot: ResourceSnapshot, +): RemoveState { + return async function* () { + const store = yield* step.store(resourceStateStore, { + id: opts.state.storeId(resource.id), + }); + const result = yield* store.deleteFrom( + `state:delete:${resource.id}`, + snapshot, + ); + if (!result.deleted) { + // "conflict" carries the version the record moved to; "not-found" means + // the record is genuinely gone, which the message reports as "missing". + throw new VersionConflict( + resource.id, + snapshot.version, + result.reason === "conflict" ? result.actualVersion : undefined, + ); + } + }; +} diff --git a/packages/reconciler/src/durable/state-backend.ts b/packages/reconciler/src/durable/state-backend.ts new file mode 100644 index 0000000..092ef85 --- /dev/null +++ b/packages/reconciler/src/durable/state-backend.ts @@ -0,0 +1,69 @@ +import type { StateNode } from "@notation/state"; +import { + resourceStateStore, + toStateNode, + type ResourceSnapshot, +} from "./stores"; +import type { StoreClient } from "./yieldstar"; + +/** + * Reads deployment state from outside a workflow, for the planner and for + * anything reporting on a deployment. Read-only: state writes must be stamped + * with the workflow step that made them, and this interface has nowhere to + * carry that step key, so a write made here would repeat on replay. + */ +export class DurableStateBackend { + /** The deployment this backend belongs to; workflows and the deployment + * hold key off this rather than carrying the ID separately. */ + readonly deploymentId: string; + readonly #client: StoreClient; + readonly #prefix: string; + + constructor(client: StoreClient, deploymentId: string) { + this.deploymentId = deploymentId; + this.#client = client; + // Keep deployment prefixes disjoint so orphan cleanup cannot delete + // another deployment's stores. + this.#prefix = `${encodeURIComponent(deploymentId)}:`; + } + + storeId(resourceId: string) { + return `${this.#prefix}${resourceId}`; + } + + async get(id: string): Promise { + const snapshot = await this.snapshot(id); + return snapshot ? toStateNode(snapshot) : undefined; + } + + snapshot(id: string): Promise { + return this.#read(this.storeId(id)); + } + + async values(): Promise { + const ids = await this.#client.listStores(resourceStateStore); + const snapshots = await Promise.all( + ids + .filter((id) => id.startsWith(this.#prefix)) + .map((id) => this.#read(id)), + ); + return snapshots + .filter((snapshot) => snapshot !== undefined) + .map(toStateNode); + } + + // getStore throws for a missing store, and the error is indistinguishable + // from a real failure, so absence is confirmed by listing. + async #read(storeId: string): Promise { + try { + return await this.#client.getStore({ + definition: resourceStateStore, + id: storeId, + }); + } catch (error) { + const ids = await this.#client.listStores(resourceStateStore); + if (!ids.includes(storeId)) return undefined; + throw error; + } + } +} diff --git a/packages/reconciler/src/durable/step.ts b/packages/reconciler/src/durable/step.ts new file mode 100644 index 0000000..b3ec055 --- /dev/null +++ b/packages/reconciler/src/durable/step.ts @@ -0,0 +1,67 @@ +import type { + EmitStep, + ReconcilerEvent, + ReconcilerEventEmitter, +} from "../events"; +import type { StepRunner } from "../operations"; +import type { DurableStep } from "./yieldstar"; + +/** + * The operation seam (`StepRunner`) plus the store handle, which only the + * durable driver uses. `DurableStep` is yieldstar's raw step primitive; + * `scopeStep` wraps one of those into one of these. + */ +export interface DurableStepRunner extends StepRunner { + /** Narrower than StepRunner's, so a scope keeps its store handle. */ + scope(prefix: string): DurableStepRunner; + store: DurableStep["store"]; +} + +/** + * Namespaces the step keys of `step` so an operation can be written once and + * replayed at several call sites without its keys colliding. + * + * Opening a store is not prefixed: yieldstar derives that key from the store + * name and store id, which is already unique. The keys a store *handle* takes + * are caller-supplied and are left alone too — a store outlives the scope + * that opened it, so its call sites qualify their own keys. + */ +export function scopeStep( + step: DurableStep, + prefix: string, +): DurableStepRunner { + const scoped = (key: string) => `${prefix}:${key}`; + + return { + run: ((key: string, fn: any) => + step.run(scoped(key), fn)) as DurableStepRunner["run"], + delay: (key: string, ms: number) => step.delay(scoped(key), ms), + store: step.store, + scope: (nested: string) => scopeStep(step, scoped(nested)), + }; +} + +/** + * Checkpoints delivery so that replaying a workflow does not re-emit. The key + * is derived from the event itself, which keeps it deterministic across a + * replay; the enclosing scope is what keeps it unique, since an operation + * emits each (operation, status) pair at most once. + * + * Emitters must still tolerate a duplicate: the process can crash after the + * event is delivered but before the checkpoint is written. + */ +export function durableEmitter( + step: Pick, + emit: ReconcilerEventEmitter | undefined, +): EmitStep { + return async function* (event) { + if (!emit) return; + yield* step.run(emitKey(event), () => emit(event)); + }; +} + +function emitKey(event: ReconcilerEvent): string { + return event.event === "reconciler.operation.lifecycle" + ? `emit:${event.event}:${event.operation}:${event.status}` + : `emit:${event.event}`; +} diff --git a/packages/reconciler/src/durable/stores.ts b/packages/reconciler/src/durable/stores.ts new file mode 100644 index 0000000..7ec3e8a --- /dev/null +++ b/packages/reconciler/src/durable/stores.ts @@ -0,0 +1,54 @@ +import type { StateNode } from "@notation/state"; +import * as v from "valibot"; +import type { PersistedResourceState } from "../operations"; +import { defineStore, type StoreSnapshot } from "./yieldstar"; + +// Store names are persisted identifiers, like the step keys mapped in +// index.ts: renaming one orphans every record stored under the old name. +// Notation-owned store names carry the "notation/" prefix, because an +// application may share a store client with these workflows. + +/** + * `looseObject` because PersistedResourceState carries an index signature: a + * driver may persist fields this schema does not name, and `v.object` would + * strip them at the store boundary. + */ +export const resourceStateStore = defineStore( + "notation/resource-state", + v.looseObject({ + id: v.string(), + type: v.string(), + // -1 and "" are BaseResource's defaults for a resource with no group. + groupId: v.number(), + groupType: v.string(), + config: v.record(v.string(), v.unknown()), + params: v.record(v.string(), v.unknown()), + output: v.record(v.string(), v.unknown()), + // Only the operations that leave a record behind: delete removes the + // store, and drift repair persists as "update". + lastOperation: v.picklist(["create", "update"]), + lastOperationAt: v.string(), + }), +); + +export const deploymentHoldStore = defineStore( + "notation/deployment-hold", + v.object({ holder: v.nullable(v.string()) }), +); + +// The schema validates exactly the record operations persist; drift between +// the two is a type error here. +({}) as v.InferOutput< + typeof resourceStateStore.schema +> satisfies PersistedResourceState; + +export type DeploymentHoldState = v.InferOutput< + typeof deploymentHoldStore.schema +>; + +/** A read of a resource record, carrying the identity a write is made against. */ +export type ResourceSnapshot = StoreSnapshot; + +export function toStateNode(snapshot: ResourceSnapshot): StateNode { + return { ...snapshot.state, version: snapshot.version }; +} diff --git a/packages/reconciler/src/durable/types.ts b/packages/reconciler/src/durable/types.ts new file mode 100644 index 0000000..3679ca8 --- /dev/null +++ b/packages/reconciler/src/durable/types.ts @@ -0,0 +1,18 @@ +import type { BaseResource } from "@notation/resource"; +import type { ReconcilerEventEmitter } from "../events"; +import type { ResourceRegistry } from "../resource-registry"; +import type { DurableStateBackend } from "./state-backend"; + +export type DurableWorkflowOptions = { + executionId: string; + resources: BaseResource[]; + state: DurableStateBackend; + registry?: ResourceRegistry; + dryRun?: boolean; + emit?: ReconcilerEventEmitter; + maxOperationAttempts?: number; +}; + +export type DurableDeployOptions = DurableWorkflowOptions & { + driftDetection?: boolean; +}; diff --git a/packages/reconciler/src/durable/yieldstar.ts b/packages/reconciler/src/durable/yieldstar.ts new file mode 100644 index 0000000..afa9871 --- /dev/null +++ b/packages/reconciler/src/durable/yieldstar.ts @@ -0,0 +1,8 @@ +import type { WorkflowFn } from "yieldstar"; + +export { defineStore } from "yieldstar"; +export type { WorkflowStore } from "yieldstar"; +export type { StoreClient, StoreSnapshot } from "@yieldstar/core"; + +/** The durable step primitive the runtime hands to workflow functions. */ +export type DurableStep = Parameters>[0]; diff --git a/packages/reconciler/src/events.ts b/packages/reconciler/src/events.ts new file mode 100644 index 0000000..df93c66 --- /dev/null +++ b/packages/reconciler/src/events.ts @@ -0,0 +1,85 @@ +import type { ResourceType } from "@notation/resource"; + +export type OperationName = "create" | "read" | "update" | "delete"; + +export type OperationLifecycleStatus = + "start" | "success" | "error" | "skip" | "dry-run"; + +export type OperationLifecycleEvent = { + level: "info" | "error"; + event: "reconciler.operation.lifecycle"; + operation: OperationName; + status: OperationLifecycleStatus; + resourceId: string; + resourceType: ResourceType; + reason?: string; + errorName?: string; + errorMessage?: string; +}; + +export type DeployDecisionEvent = { + level: "info"; + event: "reconciler.deploy.decision"; + resourceId: string; + resourceType: ResourceType; + decision: "create" | "update" | "drift-update" | "drift-recreate" | "noop"; +}; + +export type DriftDetectedEvent = { + level: "info"; + event: "reconciler.drift.detected"; + resourceId: string; + resourceType: ResourceType; + diff: Record; +}; + +export type HoldWaitingEvent = { + level: "warn"; + event: "reconciler.hold.waiting"; + deploymentId: string; + executionId: string; + holderExecutionId: string; +}; + +export type OrphanDeletionSkippedEvent = { + level: "warn"; + event: "reconciler.orphan-deletion.skipped"; + reason: "resource-type-not-registered"; + workflow: "deploy" | "destroy"; + resourceId: string; + resourceType: ResourceType; +}; + +export type ReconcilerEvent = + | OperationLifecycleEvent + | DeployDecisionEvent + | DriftDetectedEvent + | HoldWaitingEvent + | OrphanDeletionSkippedEvent; + +export type ReconcilerEventEmitter = ( + event: ReconcilerEvent, +) => void | Promise; + +/** + * The seam a driver fills in to deliver an event. Emission is a step so that + * each driver decides how it is recorded: `toEmitStep` simply awaits the + * emitter, while the durable driver's `durableEmitter` checkpoints it so that + * replaying a workflow does not re-emit events it has already delivered. + */ +export type EmitStep = ( + event: TEvent, +) => AsyncGenerator; + +/** + * Adapts a plain emitter to the driver seam, for drivers that just await. + * Absorbs an absent emitter: the returned step then delivers nothing, so + * downstream code always has an emit step and never guards. + */ +export function toEmitStep( + emit?: (event: TEvent) => void | Promise, +): EmitStep { + return async function* (event) { + await emit?.(event); + }; +} diff --git a/packages/reconciler/src/index.ts b/packages/reconciler/src/index.ts index 44fc402..b0b90e5 100644 --- a/packages/reconciler/src/index.ts +++ b/packages/reconciler/src/index.ts @@ -1,12 +1,9 @@ -export type ResourceApi = typeof import("@notation/resource"); -export type StateApi = typeof import("@notation/state"); -export type DeepObjectDiffApi = typeof import("deep-object-diff"); -export type YieldstarApi = typeof import("yieldstar"); - export * from "./resource-registry"; export * from "./operations"; export * from "./dependency-graph"; +export * from "./events"; export * from "./plan"; -export * from "./reconciler"; +export * from "./planner"; +export * from "./step-runner"; export * from "./logger-subscriber"; export * from "./protocol"; diff --git a/packages/reconciler/src/logger-subscriber.ts b/packages/reconciler/src/logger-subscriber.ts index 318a4cb..e756ff8 100644 --- a/packages/reconciler/src/logger-subscriber.ts +++ b/packages/reconciler/src/logger-subscriber.ts @@ -1,4 +1,4 @@ -import type { ReconcilerEvent, ReconcilerEventEmitter } from "./reconciler"; +import type { ReconcilerEvent, ReconcilerEventEmitter } from "./events"; export type Logger = Pick; @@ -17,7 +17,7 @@ export function createLoggerReconcilerSubscriber( return; } - if (event.event === "reconciler.orphan-deletion.skipped") { + if (event.level === "warn") { logger.warn(event.event, event); return; } diff --git a/packages/reconciler/src/operations/operation.create.ts b/packages/reconciler/src/operations/operation.create.ts index e0aee4f..2aa60b4 100644 --- a/packages/reconciler/src/operations/operation.create.ts +++ b/packages/reconciler/src/operations/operation.create.ts @@ -1,4 +1,3 @@ -import { createWorkflow } from "yieldstar"; import { type CreateResourceParams, type StepRunner, @@ -12,26 +11,22 @@ export async function* createResourceOperation( step: StepRunner, params: CreateResourceParams, ): AsyncGenerator { - await emitLifecycleEvent(params, "create", "start"); + yield* emitLifecycleEvent(params, "create", "start"); if (params.dryRun) { - await emitLifecycleEvent(params, "create", "dry-run"); + yield* emitLifecycleEvent(params, "create", "dry-run"); return; } try { - const resourceParams = yield* step.run("create:get-params", () => - params.resource.getParams(), - ); - const computedPrimaryKey = yield* runPendingOperation( step, "create:remote", - (context) => params.resource.create(resourceParams, context), + (context) => params.resource.create(params.resourceParams, context), params.maxOperationAttempts, ); - params.resource.setOutput(resourceParams); + params.resource.setOutput(params.resourceParams); if (computedPrimaryKey) { params.resource.setOutput({ ...computedPrimaryKey, @@ -41,7 +36,8 @@ export async function* createResourceOperation( const readResult = yield* readResourceOperation(step, { resource: params.resource, - state: params.state, + resourceParams: params.resourceParams, + persistedOutput: params.persistedOutput, emit: params.emit, maxOperationAttempts: params.maxOperationAttempts, }); @@ -51,32 +47,21 @@ export async function* createResourceOperation( ...readResult, }); - yield* step.run("create:persist-state", async () => { - await params.state.update(params.resource.id, params.expectedRev, { - id: params.resource.id, - groupId: params.resource.groupId, - groupType: params.resource.groupType, - type: params.resource.type, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - config: params.resource.config, - params: params.resource.toState(resourceParams), - output: params.resource.toState(params.resource.output), - }); + yield* params.persist({ + id: params.resource.id, + groupId: params.resource.groupId, + groupType: params.resource.groupType, + type: params.resource.type, + lastOperation: "create", + lastOperationAt: new Date().toISOString(), + config: params.resource.config, + params: params.resource.toState(params.resourceParams), + output: params.resource.toState(params.resource.output), }); - await emitLifecycleEvent(params, "create", "success"); + yield* emitLifecycleEvent(params, "create", "success"); } catch (err) { - await emitLifecycleEvent(params, "create", "error", getErrorDetails(err)); + yield* emitLifecycleEvent(params, "create", "error", getErrorDetails(err)); throw err; } } - -export const createResourceWorkflow: unknown = createWorkflow( - async function* (step, event) { - return yield* createResourceOperation( - step as StepRunner, - event.params as CreateResourceParams, - ); - }, -); diff --git a/packages/reconciler/src/operations/operation.delete.ts b/packages/reconciler/src/operations/operation.delete.ts index 51975b6..44f6558 100644 --- a/packages/reconciler/src/operations/operation.delete.ts +++ b/packages/reconciler/src/operations/operation.delete.ts @@ -1,4 +1,4 @@ -import { createWorkflow } from "yieldstar"; +import { ResourceNotFoundError } from "@notation/resource"; import { type DeleteResourceParams, type StepRunner, @@ -11,42 +11,38 @@ export async function* deleteResourceOperation( step: StepRunner, params: DeleteResourceParams, ): AsyncGenerator { - await emitLifecycleEvent(params, "delete", "start"); + yield* emitLifecycleEvent(params, "delete", "start"); if (params.dryRun) { - await emitLifecycleEvent(params, "delete", "dry-run"); + yield* emitLifecycleEvent(params, "delete", "dry-run"); return; } try { - yield* runPendingOperation( - step, - "delete:remote", - (context) => - params.resource.delete( - params.resource.key, - params.resource.toState(params.resource.output), - context, - ), - params.maxOperationAttempts, - ); + try { + yield* runPendingOperation( + step, + "delete:remote", + (context) => + params.resource.delete( + params.resource.key, + params.resource.toState(params.resource.output), + context, + ), + params.maxOperationAttempts, + ); + } catch (error) { + // Absence is delete's goal state, so a delete that finds the resource + // already gone — a crash-window replay, or an out-of-band removal — + // has succeeded. + if (!ResourceNotFoundError.is(error)) throw error; + } - yield* step.run("delete:persist-state", () => - params.state.delete(params.resource.id, params.expectedRev), - ); + yield* params.remove(); - await emitLifecycleEvent(params, "delete", "success"); + yield* emitLifecycleEvent(params, "delete", "success"); } catch (err) { - await emitLifecycleEvent(params, "delete", "error", getErrorDetails(err)); + yield* emitLifecycleEvent(params, "delete", "error", getErrorDetails(err)); throw err; } } - -export const deleteResourceWorkflow: unknown = createWorkflow( - async function* (step, event) { - return yield* deleteResourceOperation( - step as StepRunner, - event.params as DeleteResourceParams, - ); - }, -); diff --git a/packages/reconciler/src/operations/operation.read.ts b/packages/reconciler/src/operations/operation.read.ts index e74196a..72705be 100644 --- a/packages/reconciler/src/operations/operation.read.ts +++ b/packages/reconciler/src/operations/operation.read.ts @@ -1,6 +1,7 @@ -import { createWorkflow } from "yieldstar"; +import { ResourceNotFoundError } from "@notation/resource"; +import { decideDriftAction, type DriftRead, type ResourceAction } from "../plan"; import { - type ReadResourceParams, + type ResolvedResourceParams, type StepRunner, emitLifecycleEvent, getErrorDetails, @@ -9,32 +10,25 @@ import { runPendingOperation } from "./operation.pending"; export async function* readResourceOperation( step: StepRunner, - params: ReadResourceParams, + params: ResolvedResourceParams, ): AsyncGenerator, unknown> { - await emitLifecycleEvent(params, "read", "start"); + yield* emitLifecycleEvent(params, "read", "start"); if (params.dryRun) { - await emitLifecycleEvent(params, "read", "dry-run"); + yield* emitLifecycleEvent(params, "read", "dry-run"); return {}; } try { - const resourceParams = yield* step.run("read:get-params", () => - params.resource.getParams(), - ); - if (!params.resource.read) { - const stateNode = yield* step.run("read:get-state-node", () => - params.state.get(params.resource.id), - ); - const merged = stateNode - ? { ...stateNode.output, ...resourceParams } - : resourceParams; + const merged = params.persistedOutput + ? { ...params.persistedOutput, ...params.resourceParams } + : params.resourceParams; - await emitLifecycleEvent(params, "read", "skip", { + yield* emitLifecycleEvent(params, "read", "skip", { reason: "read-not-implemented", }); - await emitLifecycleEvent(params, "read", "success"); + yield* emitLifecycleEvent(params, "read", "success"); return merged as Record; } @@ -46,23 +40,63 @@ export async function* readResourceOperation( ); const mergedOutput = { - ...resourceParams, + ...params.resourceParams, ...remote, }; - await emitLifecycleEvent(params, "read", "success"); + yield* emitLifecycleEvent(params, "read", "success"); return mergedOutput; } catch (err) { - await emitLifecycleEvent(params, "read", "error", getErrorDetails(err)); + yield* emitLifecycleEvent(params, "read", "error", getErrorDetails(err)); throw err; } } -export const readResourceWorkflow: unknown = createWorkflow( - async function* (step, event) { - return yield* readResourceOperation( - step as StepRunner, - event.params as ReadResourceParams, - ); +/** + * Reads the remote to compare it against persisted state. An absent resource + * is a fact about the world rather than a failure, so it is reported as such; + * every other error still propagates. + */ +export async function* readDriftOperation( + step: StepRunner, + params: ResolvedResourceParams, +): AsyncGenerator { + try { + const output = yield* readResourceOperation(step, params); + return { kind: "present", output }; + } catch (error) { + if (ResourceNotFoundError.is(error)) return { kind: "absent" }; + throw error; + } +} + +/** + * The drift gate, shared by every driver: a noop is only trusted once the + * remote has been read back, because the provider may have drifted from + * persisted state, which upgrades the decision. A resource with no read has + * no remote to compare, so its noop stands. `driftDetection` defaults to on + * here and nowhere else. Any other decision passes through untouched. + */ +export async function* applyDriftDetection( + step: StepRunner, + params: ResolvedResourceParams & { + action: ResourceAction; + driftDetection?: boolean; }, -); +): AsyncGenerator { + const { action, driftDetection, ...readParams } = params; + if ( + action.decision !== "noop" || + !(driftDetection ?? true) || + !readParams.resource.read + ) { + return action; + } + + const driftRead = yield* readDriftOperation(step, readParams); + return decideDriftAction({ + resource: readParams.resource, + params: readParams.resourceParams, + driftRead, + }); +} diff --git a/packages/reconciler/src/operations/operation.types.ts b/packages/reconciler/src/operations/operation.types.ts index 6cbcd1c..9f9c8ef 100644 --- a/packages/reconciler/src/operations/operation.types.ts +++ b/packages/reconciler/src/operations/operation.types.ts @@ -1,58 +1,96 @@ -import type { BaseResource, ResourceType } from "@notation/resource"; -import type { State } from "@notation/state"; +import type { BaseResource } from "@notation/resource"; +import type { StateNode } from "@notation/state"; +import type { + EmitStep, + OperationLifecycleEvent, + OperationLifecycleStatus, + OperationName, +} from "../events"; -export type OperationName = "create" | "read" | "update" | "delete"; - -export type OperationLifecycleStatus = - "start" | "success" | "error" | "skip" | "dry-run"; - -export type OperationLifecycleEvent = { - level: "info" | "error"; - event: "reconciler.operation.lifecycle"; - operation: OperationName; - status: OperationLifecycleStatus; - resourceId: string; - resourceType: ResourceType; - reason?: string; - errorName?: string; - errorMessage?: string; -}; - -export type OperationEventEmitter = ( - event: OperationLifecycleEvent, -) => void | Promise; +export type { + OperationLifecycleEvent, + OperationLifecycleStatus, + OperationName, +} from "../events"; +/** + * How an operation runs a step. Keys identify a step's cached result across a + * replay; `scope` namespaces them so one operation can run at several call + * sites in a single execution. `createStepRunner` ignores both. + */ export type StepRunner = { - run(fn: () => T | Promise): AsyncGenerator; run( key: string, fn: () => T | Promise, ): AsyncGenerator; - delay(ms: number): AsyncGenerator; delay(key: string, ms: number): AsyncGenerator; + scope(prefix: string): StepRunner; +}; + +/** + * The record an operation wants persisted; the driver owns the version. + * Not derived with Omit, which would collapse against StateNode's index + * signature and widen every field to unknown. + */ +export type PersistedResourceState = Pick< + StateNode, + | "id" + | "type" + | "config" + | "params" + | "output" + | "lastOperation" + | "lastOperationAt" +> & { + // Not on StateNode itself, where they arrive through its index signature. + groupId: number; + groupType: string; + [key: string]: unknown; }; +/** + * How the driver writes state. Both are steps so the driver can carry its own + * concurrency control: in a workflow that is a store write stamped with the + * step that made it, so a replay does not repeat it. + */ +export type PersistState = ( + next: PersistedResourceState, +) => AsyncGenerator; + +export type RemoveState = () => AsyncGenerator; + export type ResourceOperationBaseParams = { resource: BaseResource; - state: Pick; dryRun?: boolean; - emit?: OperationEventEmitter; + /** Always present: an absent emitter is absorbed where the step is made + * (`toEmitStep`, `durableEmitter`), not guarded here. */ + emit: EmitStep; maxOperationAttempts?: number; }; -export type CreateResourceParams = ResourceOperationBaseParams & { - expectedRev: number; +/** + * Inputs resolved before an operation starts: the desired params, and — for + * a resource with no read operation — the output the last write persisted. + * An operation that resolved either itself could see a different answer from + * the one the decision was taken against. Read takes exactly this; create + * and update add their write. + */ +export type ResolvedResourceParams = ResourceOperationBaseParams & { + resourceParams: Record; + persistedOutput?: Record; }; -export type ReadResourceParams = ResourceOperationBaseParams; +export type CreateResourceParams = ResolvedResourceParams & { + persist: PersistState; +}; -export type UpdateResourceParams = ResourceOperationBaseParams & { +export type UpdateResourceParams = ResolvedResourceParams & { patch: Record; - expectedRev: number; + persist: PersistState; }; export type DeleteResourceParams = ResourceOperationBaseParams & { - expectedRev: number; + remove: RemoveState; }; export function getErrorDetails(err: unknown): { @@ -72,15 +110,13 @@ export function getErrorDetails(err: unknown): { }; } -export async function emitLifecycleEvent( +export async function* emitLifecycleEvent( params: ResourceOperationBaseParams, operation: OperationName, status: OperationLifecycleStatus, extra: Partial = {}, -) { - if (!params.emit) return; - - await params.emit({ +): AsyncGenerator { + yield* params.emit({ level: status === "error" ? "error" : "info", event: "reconciler.operation.lifecycle", operation, diff --git a/packages/reconciler/src/operations/operation.update.ts b/packages/reconciler/src/operations/operation.update.ts index fc62a8c..ca51010 100644 --- a/packages/reconciler/src/operations/operation.update.ts +++ b/packages/reconciler/src/operations/operation.update.ts @@ -1,4 +1,3 @@ -import { createWorkflow } from "yieldstar"; import { type StepRunner, type UpdateResourceParams, @@ -12,26 +11,22 @@ export async function* updateResourceOperation( step: StepRunner, params: UpdateResourceParams, ): AsyncGenerator { - await emitLifecycleEvent(params, "update", "start"); + yield* emitLifecycleEvent(params, "update", "start"); if (params.dryRun) { - await emitLifecycleEvent(params, "update", "dry-run"); + yield* emitLifecycleEvent(params, "update", "dry-run"); return; } if (!params.resource.update) { - await emitLifecycleEvent(params, "update", "skip", { + yield* emitLifecycleEvent(params, "update", "skip", { reason: "update-not-implemented", }); - await emitLifecycleEvent(params, "update", "success"); + yield* emitLifecycleEvent(params, "update", "success"); return; } try { - const resourceParams = yield* step.run("update:get-params", () => - params.resource.getParams(), - ); - yield* runPendingOperation( step, "update:remote", @@ -39,7 +34,7 @@ export async function* updateResourceOperation( params.resource.update!( params.resource.key, params.patch, - resourceParams, + params.resourceParams, params.resource.toState(params.resource.output), context, ), @@ -48,12 +43,13 @@ export async function* updateResourceOperation( params.resource.setOutput({ ...params.resource.key, - ...resourceParams, + ...params.resourceParams, }); const readResult = yield* readResourceOperation(step, { resource: params.resource, - state: params.state, + resourceParams: params.resourceParams, + persistedOutput: params.persistedOutput, emit: params.emit, maxOperationAttempts: params.maxOperationAttempts, }); @@ -63,32 +59,21 @@ export async function* updateResourceOperation( ...readResult, }); - yield* step.run("update:persist-state", async () => { - await params.state.update(params.resource.id, params.expectedRev, { - id: params.resource.id, - groupId: params.resource.groupId, - groupType: params.resource.groupType, - type: params.resource.type, - lastOperation: "update", - lastOperationAt: new Date().toISOString(), - config: params.resource.config, - params: params.resource.toState(resourceParams), - output: params.resource.toState(params.resource.output), - }); + yield* params.persist({ + id: params.resource.id, + groupId: params.resource.groupId, + groupType: params.resource.groupType, + type: params.resource.type, + lastOperation: "update", + lastOperationAt: new Date().toISOString(), + config: params.resource.config, + params: params.resource.toState(params.resourceParams), + output: params.resource.toState(params.resource.output), }); - await emitLifecycleEvent(params, "update", "success"); + yield* emitLifecycleEvent(params, "update", "success"); } catch (err) { - await emitLifecycleEvent(params, "update", "error", getErrorDetails(err)); + yield* emitLifecycleEvent(params, "update", "error", getErrorDetails(err)); throw err; } } - -export const updateResourceWorkflow: unknown = createWorkflow( - async function* (step, event) { - return yield* updateResourceOperation( - step as StepRunner, - event.params as UpdateResourceParams, - ); - }, -); diff --git a/packages/reconciler/src/plan.ts b/packages/reconciler/src/plan.ts index d2f8451..1b7dfb3 100644 --- a/packages/reconciler/src/plan.ts +++ b/packages/reconciler/src/plan.ts @@ -51,55 +51,20 @@ export type ResourceAction = export function decideAction(opts: { resource: BaseResource; stateNode?: StateNode; - params?: Record; - driftRead?: DriftRead; + params: Record; }): ResourceAction { - const { resource, stateNode, params, driftRead } = opts; - const desiredComparable = resource.toComparable(params ?? {}); - const previousComparable = resource.toComparable(stateNode?.params ?? {}); + const { resource, stateNode, params } = opts; + if (!stateNode) { + return { decision: "create" }; + } + + const desiredComparable = resource.toComparable(params); + const previousComparable = resource.toComparable(stateNode.params); const localPatch = diff(previousComparable, desiredComparable) as Record< string, unknown >; - if (driftRead) { - if (driftRead.kind === "absent") { - return { decision: stateNode ? "drift-recreate" : "create" }; - } - - const remoteComparable = resource.toComparable(driftRead.output); - const remotePatch = diff(remoteComparable, desiredComparable) as Record< - string, - unknown - >; - - if (Object.keys(remotePatch).length === 0) { - return { decision: "noop" }; - } - - const remoteDetailedDiff = detailedDiff( - remoteComparable, - desiredComparable, - ); - if (!stateNode || Object.keys(localPatch).length > 0) { - return { - decision: "update", - patch: remotePatch, - diff: toPlanDiff(remoteDetailedDiff), - }; - } - - return { - decision: "drift-update", - patch: remotePatch, - diff: toPlanDiff(remoteDetailedDiff), - }; - } - - if (!stateNode) { - return { decision: "create" }; - } - if (Object.keys(localPatch).length > 0) { return { decision: "update", @@ -111,6 +76,39 @@ export function decideAction(opts: { return { decision: "noop" }; } +/** + * Upgrades a noop decision with a read of the remote. Callers reach this only + * after decideAction returned noop, so a state node exists and the desired + * params match it: the remote is the only remaining source of difference. + */ +export function decideDriftAction(opts: { + resource: BaseResource; + params: Record; + driftRead: DriftRead; +}): ResourceAction { + const { resource, params, driftRead } = opts; + if (driftRead.kind === "absent") { + return { decision: "drift-recreate" }; + } + + const desiredComparable = resource.toComparable(params); + const remoteComparable = resource.toComparable(driftRead.output); + const remotePatch = diff(remoteComparable, desiredComparable) as Record< + string, + unknown + >; + + if (Object.keys(remotePatch).length === 0) { + return { decision: "noop" }; + } + + return { + decision: "drift-update", + patch: remotePatch, + diff: toPlanDiff(detailedDiff(remoteComparable, desiredComparable)), + }; +} + export async function resolvePlanParams( resource: BaseResource, ): Promise> { diff --git a/packages/reconciler/src/planner.ts b/packages/reconciler/src/planner.ts new file mode 100644 index 0000000..441d249 --- /dev/null +++ b/packages/reconciler/src/planner.ts @@ -0,0 +1,76 @@ +import type { BaseResource } from "@notation/resource"; +import type { StateBackend } from "@notation/state"; +import { buildResourceDepthLevels } from "./dependency-graph"; +import { toEmitStep, type ReconcilerEventEmitter } from "./events"; +import { applyDriftDetection } from "./operations"; +import { + decideAction, + getDependencyIds, + resolvePlanParams, + type Plan, + type PlanNode, +} from "./plan"; +import { createStepRunner, runOperation } from "./step-runner"; + +export type CreatePlanOptions = { + resources: BaseResource[]; + state: StateBackend; + driftDetection?: boolean; + emit?: ReconcilerEventEmitter; + maxOperationAttempts?: number; +}; + +export async function createPlan({ + resources, + state, + driftDetection, + emit, + maxOperationAttempts, +}: CreatePlanOptions): Promise { + const resourceById = new Map( + resources.map((resource) => [resource.id, resource]), + ); + const emitStep = toEmitStep(emit); + const nodes: PlanNode[] = []; + + for (const level of buildResourceDepthLevels(resources)) { + for (const resource of level) { + const stateNode = await state.get(resource.id); + if (stateNode) resource.setOutput(stateNode.output); + const params = await resolvePlanParams(resource); + const action = await runOperation( + applyDriftDetection(createStepRunner(), { + action: decideAction({ resource, stateNode, params }), + driftDetection, + resource, + resourceParams: params, + persistedOutput: stateNode?.output, + emit: emitStep, + maxOperationAttempts, + }), + ); + + nodes.push({ + id: resource.id, + type: resource.type, + decision: action.decision, + ...("diff" in action ? { diff: action.diff } : {}), + params, + dependsOn: getDependencyIds(resource), + }); + } + } + + for (const stateNode of await state.values()) { + if (resourceById.has(stateNode.id)) continue; + nodes.push({ + id: stateNode.id, + type: stateNode.type, + decision: "delete-orphan", + params: stateNode.params, + dependsOn: [], + }); + } + + return { createdAt: new Date().toISOString(), nodes }; +} diff --git a/packages/reconciler/src/protocol.ts b/packages/reconciler/src/protocol.ts index bf3706e..a8ab407 100644 --- a/packages/reconciler/src/protocol.ts +++ b/packages/reconciler/src/protocol.ts @@ -1,4 +1,4 @@ -import type { ReconcilerEvent, ReconcilerEventEmitter } from "./reconciler"; +import type { ReconcilerEvent, ReconcilerEventEmitter } from "./events"; export const EVENT_STREAM_VERSION = 1 as const; diff --git a/packages/reconciler/src/reconciler.ts b/packages/reconciler/src/reconciler.ts deleted file mode 100644 index fab6c0b..0000000 --- a/packages/reconciler/src/reconciler.ts +++ /dev/null @@ -1,602 +0,0 @@ -import { ResourceNotFoundError } from "@notation/resource"; -import type { BaseResource, ResourceType } from "@notation/resource"; -import { RevConflict, type State, type StateNode } from "@notation/state"; -import { setTimeout as sleep } from "node:timers/promises"; -import { buildResourceDepthLevels } from "./dependency-graph"; -import { - decideAction, - getDependencyIds, - resolvePlanParams, - type DriftRead, - type Plan, - type PlanNode, - type ResourceAction, -} from "./plan"; -import { - createResourceOperation, - deleteResourceOperation, - readResourceOperation, - type OperationLifecycleEvent, - type StepRunner, - updateResourceOperation, -} from "./operations"; -import { - createMissingResourceRegistryMatchWarningEvent, - createResourceRegistryFromResources, - resolveResourceClass, - type MissingResourceRegistryMatchWarningEvent, - type ResourceRegistry, -} from "./resource-registry"; - -export type ReconcilerDeployEvent = { - level: "info"; - event: "reconciler.deploy.decision"; - resourceId: string; - resourceType: string; - decision: "create" | "update" | "drift-update" | "drift-recreate" | "noop"; -}; - -export type ReconcilerDriftDetectedEvent = { - level: "info"; - event: "reconciler.drift.detected"; - resourceId: string; - resourceType: string; - diff: Record; -}; - -export type ReconcilerEvent = - | OperationLifecycleEvent - | ReconcilerDeployEvent - | ReconcilerDriftDetectedEvent - | MissingResourceRegistryMatchWarningEvent; - -export type ReconcilerEventEmitter = ( - event: ReconcilerEvent, -) => void | Promise; - -export type ReconcilerState = Pick< - State, - "get" | "update" | "delete" | "values" | "lease" ->; - -export type ReconcilerOptions = { - state: ReconcilerState; - registry?: ResourceRegistry; - dryRun?: boolean; - driftDetection?: boolean; - emit?: ReconcilerEventEmitter; - maxOperationAttempts?: number; - mutationLeaseTtl?: number; -}; - -export type DeployOptions = { - dryRun?: boolean; - driftDetection?: boolean; -}; - -export type DestroyOptions = { - dryRun?: boolean; -}; - -export type RefreshOptions = { - dryRun?: boolean; -}; - -export type PlanOptions = { - driftDetection?: boolean; -}; - -export class Reconciler { - readonly #state: ReconcilerState; - readonly #registry?: ResourceRegistry; - readonly #defaultDryRun: boolean; - readonly #defaultDriftDetection: boolean; - readonly #emit?: ReconcilerEventEmitter; - readonly #maxOperationAttempts?: number; - readonly #mutationLeaseTtl: number; - readonly #stepRunner: StepRunner; - - constructor(opts: ReconcilerOptions) { - this.#state = opts.state; - this.#registry = opts.registry; - this.#defaultDryRun = opts.dryRun ?? false; - this.#defaultDriftDetection = opts.driftDetection ?? true; - this.#emit = opts.emit; - this.#maxOperationAttempts = opts.maxOperationAttempts; - this.#mutationLeaseTtl = opts.mutationLeaseTtl ?? 30_000; - this.#stepRunner = createStepRunner(); - } - - async deploy( - resources: BaseResource[], - opts: DeployOptions = {}, - ): Promise { - const dryRun = opts.dryRun ?? this.#defaultDryRun; - const driftDetection = opts.driftDetection ?? this.#defaultDriftDetection; - const resourceById = new Map( - resources.map((resource) => [resource.id, resource]), - ); - - const dependencyLevels = buildResourceDepthLevels(resources); - for (const level of dependencyLevels) { - await Promise.all( - level.map((resource) => - this.#deployResource(resource, dryRun, driftDetection), - ), - ); - } - - await this.#deleteOrphans(resources, resourceById, dryRun, "deploy"); - } - - async plan(resources: BaseResource[], opts: PlanOptions = {}): Promise { - const driftDetection = opts.driftDetection ?? this.#defaultDriftDetection; - const resourceById = new Map( - resources.map((resource) => [resource.id, resource]), - ); - const nodes: PlanNode[] = []; - - const dependencyLevels = buildResourceDepthLevels(resources); - for (const level of dependencyLevels) { - for (const resource of level) { - nodes.push(await this.#planResource(resource, driftDetection)); - } - } - - const stateNodes = await this.#state.values(); - for (const stateNode of stateNodes) { - if (resourceById.has(stateNode.id)) continue; - - nodes.push({ - id: stateNode.id, - type: stateNode.type, - decision: "delete-orphan", - params: stateNode.params, - dependsOn: [], - }); - } - - return { - createdAt: new Date().toISOString(), - nodes, - }; - } - - async destroy( - resources: BaseResource[], - opts: DestroyOptions = {}, - ): Promise { - const dryRun = opts.dryRun ?? this.#defaultDryRun; - const dependencyLevels = buildResourceDepthLevels(resources); - - for ( - let levelIndex = dependencyLevels.length - 1; - levelIndex >= 0; - levelIndex -= 1 - ) { - const level = dependencyLevels[levelIndex]!; - await Promise.all( - level.map((resource) => this.#destroyResource(resource, dryRun)), - ); - } - } - - async refresh( - resources: BaseResource[], - opts: RefreshOptions = {}, - ): Promise { - const dryRun = opts.dryRun ?? this.#defaultDryRun; - const resourceById = new Map( - resources.map((resource) => [resource.id, resource]), - ); - - await this.#deleteOrphans(resources, resourceById, dryRun, "refresh"); - } - - async #deployResource( - resource: BaseResource, - dryRun: boolean, - driftDetection: boolean, - ) { - await this.#withMutationLease(resource.id, () => - this.#retryOnRevConflict((conflict) => - this.#deployResourceOnce(resource, dryRun, driftDetection, conflict), - ), - ); - } - - async #withMutationLease(resourceId: string, fn: () => Promise) { - return this.#withLease(`reconciler:resource:${resourceId}`, fn); - } - - async #withLease(scope: string, fn: () => Promise): Promise { - const lease = await this.#state.lease(scope, this.#mutationLeaseTtl); - const controller = new AbortController(); - let renewalError: unknown; - const heartbeat = (async () => { - try { - while (!controller.signal.aborted) { - await sleep( - Math.max(1, Math.floor(this.#mutationLeaseTtl / 3)), - undefined, - { - signal: controller.signal, - }, - ); - await lease.renew(this.#mutationLeaseTtl); - } - } catch (error) { - if (!controller.signal.aborted) renewalError = error; - } - })(); - - try { - const result = await fn(); - if (renewalError) throw renewalError; - return result; - } finally { - controller.abort(); - await heartbeat; - await lease.release(); - } - } - - async #retryOnRevConflict(fn: (conflict?: RevConflict) => Promise) { - let conflict: RevConflict | undefined; - for (let attempt = 0; attempt < 3; attempt += 1) { - try { - await fn(conflict); - return; - } catch (error) { - if (!(error instanceof RevConflict) || attempt === 2) throw error; - // Re-throwing the conflict supplied for recovery means the resource - // cannot be recovered safely (for example, it has no read operation). - if (error === conflict) throw error; - conflict = error; - } - } - } - - async #deployResourceOnce( - resource: BaseResource, - dryRun: boolean, - driftDetection: boolean, - conflict?: RevConflict, - ) { - if (conflict) { - await this.#recoverDeployResource(resource, dryRun, conflict); - return; - } - - const stateNode = await this.#state.get(resource.id); - - let action: ResourceAction; - if (!stateNode) { - action = decideAction({ resource }); - } else { - resource.setOutput(stateNode.output); - const params = await resource.getParams(); - action = decideAction({ resource, stateNode, params }); - - if (action.decision === "noop" && driftDetection) { - const driftRead = await this.#readForDrift(resource); - action = decideAction({ resource, stateNode, params, driftRead }); - } - } - - if (action.decision === "drift-update") { - await this.#emit?.({ - level: "info", - event: "reconciler.drift.detected", - resourceId: resource.id, - resourceType: resource.type, - diff: action.patch, - }); - } - - await this.#emit?.({ - level: "info", - event: "reconciler.deploy.decision", - resourceId: resource.id, - resourceType: resource.type, - decision: action.decision, - }); - - switch (action.decision) { - case "create": - case "drift-recreate": - await runOperation( - createResourceOperation(this.#stepRunner, { - resource, - state: this.#state, - dryRun, - emit: this.#emit, - maxOperationAttempts: this.#maxOperationAttempts, - expectedRev: stateNode?.rev ?? 0, - }), - ); - return; - case "update": - case "drift-update": - // decideAction only returns update decisions for an existing stateNode - await runOperation( - updateResourceOperation(this.#stepRunner, { - resource, - state: this.#state, - patch: action.patch, - dryRun, - emit: this.#emit, - maxOperationAttempts: this.#maxOperationAttempts, - expectedRev: stateNode!.rev, - }), - ); - return; - case "noop": - return; - } - } - - async #recoverDeployResource( - resource: BaseResource, - dryRun: boolean, - conflict: RevConflict, - ) { - if (!resource.read) throw conflict; - - const stateNode = await this.#state.get(resource.id); - if (stateNode) resource.setOutput(stateNode.output); - - const params = await resource.getParams(); - const remote = await this.#readForDrift(resource); - const action = decideAction({ - resource, - stateNode, - params, - driftRead: remote, - }); - if (remote.kind === "present") resource.setOutput(remote.output); - - await this.#emit?.({ - level: "info", - event: "reconciler.deploy.decision", - resourceId: resource.id, - resourceType: resource.type, - decision: action.decision, - }); - - switch (action.decision) { - case "create": - case "drift-recreate": - await runOperation( - createResourceOperation(this.#stepRunner, { - resource, - state: this.#state, - dryRun, - emit: this.#emit, - maxOperationAttempts: this.#maxOperationAttempts, - expectedRev: stateNode?.rev ?? 0, - }), - ); - return; - case "update": - case "drift-update": - await runOperation( - updateResourceOperation(this.#stepRunner, { - resource, - state: this.#state, - patch: action.patch, - dryRun, - emit: this.#emit, - maxOperationAttempts: this.#maxOperationAttempts, - expectedRev: stateNode?.rev ?? 0, - }), - ); - return; - case "noop": - if (dryRun) return; - await this.#state.update(resource.id, stateNode?.rev ?? 0, { - id: resource.id, - groupId: resource.groupId, - groupType: resource.groupType, - type: resource.type, - lastOperation: "drift", - lastOperationAt: new Date().toISOString(), - config: resource.config, - params: resource.toState(params), - output: resource.toState(resource.output), - }); - return; - } - } - - async #planResource( - resource: BaseResource, - driftDetection: boolean, - ): Promise { - const stateNode = await this.#state.get(resource.id); - if (stateNode) { - resource.setOutput(stateNode.output); - } - - const params = await resolvePlanParams(resource); - let action = decideAction({ resource, stateNode, params }); - - if (action.decision === "noop" && driftDetection) { - const driftRead = await this.#readForDrift(resource); - action = decideAction({ resource, stateNode, params, driftRead }); - } - - return { - id: resource.id, - type: resource.type, - decision: action.decision, - ...("diff" in action ? { diff: action.diff } : {}), - params, - dependsOn: getDependencyIds(resource), - }; - } - - async #readForDrift(resource: BaseResource): Promise { - try { - const output = await runOperation( - readResourceOperation(this.#stepRunner, { - resource, - state: this.#state, - emit: this.#emit, - maxOperationAttempts: this.#maxOperationAttempts, - }), - ); - return { kind: "present", output }; - } catch (error) { - if (ResourceNotFoundError.is(error)) return { kind: "absent" }; - throw error; - } - } - - async #deleteOrphans( - resources: BaseResource[], - resourceById: Map, - dryRun: boolean, - workflow: "deploy" | "refresh", - ) { - await this.#withLease("reconciler:orphan-deletion", async () => { - const stateNodes = await this.#state.values(); - const registry = - this.#registry ?? createResourceRegistryFromResources(resources); - - for (const stateNode of stateNodes) { - if (resourceById.has(stateNode.id)) continue; - - const stateNodeResourceType = stateNode.type as ResourceType; - - const Resource = resolveResourceClass(registry, stateNodeResourceType); - if (!Resource) { - await this.#emit?.( - createMissingResourceRegistryMatchWarningEvent({ - workflow, - resourceId: stateNode.id, - resourceType: stateNodeResourceType, - }), - ); - continue; - } - - await this.#withMutationLease(stateNode.id, () => - this.#retryOnRevConflict(async (conflict) => { - const currentNode = await this.#state.get(stateNode.id); - if (!currentNode) return; - - const orphanResource = hydrateResourceFromState( - Resource, - currentNode, - ); - - await this.#deleteResourceOnce( - orphanResource, - currentNode, - dryRun, - conflict, - ); - }), - ); - } - }); - } - - async #destroyResource(resource: BaseResource, dryRun: boolean) { - await this.#withMutationLease(resource.id, () => - this.#retryOnRevConflict(async (conflict) => { - const stateNode = await this.#state.get(resource.id); - if (!stateNode) { - return; - } - - resource.setOutput(stateNode.output); - await this.#deleteResourceOnce(resource, stateNode, dryRun, conflict); - }), - ); - } - - async #deleteResourceOnce( - resource: BaseResource, - stateNode: StateNode, - dryRun: boolean, - conflict?: RevConflict, - ) { - if (conflict) { - if (!resource.read) throw conflict; - - const remote = await this.#readForDrift(resource); - if (remote.kind !== "present") { - if (!dryRun) await this.#state.delete(resource.id, stateNode.rev); - return; - } - resource.setOutput(remote.output); - } - - await runOperation( - deleteResourceOperation(this.#stepRunner, { - resource, - state: this.#state, - dryRun, - emit: this.#emit, - maxOperationAttempts: this.#maxOperationAttempts, - expectedRev: stateNode.rev, - }), - ); - } -} - -export async function runOperation( - operation: AsyncGenerator, -) { - let next = await operation.next(); - while (!next.done) { - next = await operation.next(); - } - return next.value; -} - -function hydrateResourceFromState( - Resource: new (opts: { - id: string; - config: Record; - }) => BaseResource, - stateNode: StateNode, -): BaseResource { - const resource = new Resource({ - id: stateNode.id, - config: stateNode.config, - }); - resource.setOutput(stateNode.output); - return resource; -} - -export function createStepRunner(): StepRunner { - return { - async *run( - arg1: string | (() => T | Promise), - arg2?: () => T | Promise, - ): AsyncGenerator { - const fn = (typeof arg1 === "string" ? arg2 : arg1) as - (() => T | Promise) | undefined; - - if (!fn) { - throw new Error("Missing run function"); - } - - return await fn(); - }, - async *delay( - arg1: string | number, - arg2?: number, - ): AsyncGenerator { - const ms = typeof arg1 === "number" ? arg1 : arg2; - if (ms === undefined) { - throw new Error("Missing delay duration"); - } - - await new Promise((resolve) => setTimeout(resolve, ms)); - }, - }; -} diff --git a/packages/reconciler/src/resource-registry.ts b/packages/reconciler/src/resource-registry.ts index 916c0ab..fc39437 100644 --- a/packages/reconciler/src/resource-registry.ts +++ b/packages/reconciler/src/resource-registry.ts @@ -1,16 +1,11 @@ -import type { BaseResource, ResourceClass, ResourceType } from "@notation/resource"; +import type { + BaseResource, + ResourceClass, + ResourceType, +} from "@notation/resource"; export type ResourceRegistry = Map>; -export type MissingResourceRegistryMatchWarningEvent = { - level: "warn"; - event: "reconciler.orphan-deletion.skipped"; - reason: "resource-type-not-registered"; - workflow: "deploy" | "refresh"; - resourceId: string; - resourceType: ResourceType; -}; - export function createResourceRegistry( entries: Iterable> = [], ): ResourceRegistry { @@ -44,18 +39,3 @@ export function resolveResourceClass( ): ResourceClass | undefined { return registry.get(type); } - -export function createMissingResourceRegistryMatchWarningEvent(opts: { - workflow: "deploy" | "refresh"; - resourceId: string; - resourceType: ResourceType; -}): MissingResourceRegistryMatchWarningEvent { - return { - level: "warn", - event: "reconciler.orphan-deletion.skipped", - reason: "resource-type-not-registered", - workflow: opts.workflow, - resourceId: opts.resourceId, - resourceType: opts.resourceType, - }; -} diff --git a/packages/reconciler/src/step-runner.ts b/packages/reconciler/src/step-runner.ts new file mode 100644 index 0000000..ff39fdc --- /dev/null +++ b/packages/reconciler/src/step-runner.ts @@ -0,0 +1,27 @@ +import type { StepRunner } from "./operations"; + +export async function runOperation( + operation: AsyncGenerator, +) { + let next = await operation.next(); + while (!next.done) { + next = await operation.next(); + } + return next.value; +} + +export function createStepRunner(): StepRunner { + const runner: StepRunner = { + async *run(_key: string, fn: () => T | Promise) { + return await fn(); + }, + async *delay(_key: string, ms: number) { + await new Promise((resolve) => setTimeout(resolve, ms)); + }, + // Nothing is replayed in process, so no step key is ever read and a scope + // has nothing to namespace: one runner serves every scope. + scope: () => runner, + }; + + return runner; +} diff --git a/packages/reconciler/test/durable-reconciliation.test.ts b/packages/reconciler/test/durable-reconciliation.test.ts new file mode 100644 index 0000000..9c482d9 --- /dev/null +++ b/packages/reconciler/test/durable-reconciliation.test.ts @@ -0,0 +1,842 @@ +import { + WorkflowRunner, + type HeapClient, + type WorkflowEvent, +} from "@yieldstar/core"; +import { + SqliteHeapClient, + SqliteStoreClient, + createSqliteDb, +} from "@yieldstar/sqlite-runtime/node"; +import { + resource, + ResourceNotFoundError, + ResourceOperationPendingError, + type BaseResource, +} from "@notation/resource"; +import { setTimeout as sleep } from "node:timers/promises"; +import pino from "pino"; +import { createWorkflowRouter, workflow } from "yieldstar"; +import { describe, expect, it, vi } from "vitest"; +import * as durable from "../src/durable"; +import type { ReconcilerEvent } from "../src/events"; +import { + createResourceRegistry, + type ResourceRegistry, +} from "../src/resource-registry"; + +const logger = pino({ level: "silent" }); + +/** + * A retry delay has to outlive the heap write that follows it. The workflow + * loop continues inline for a delay that has already elapsed by the time it + * is reached, so a delay shorter than a SQLite write runs the retry in the + * same execution — which is the opposite of what these tests assert. + */ +const RETRY_AFTER_MS = 50; +const PAST_RETRY_MS = RETRY_AFTER_MS + 25; + +describe("durable execution and replay", () => { + it("waits durably for a retryable provider and persists after success", async () => { + let attempts = 0; + const PendingResource = resource({ type: "test/durable/pending" }) + .defineSchema({}) + .defineOperations({ + create: async (_params, context) => { + attempts += 1; + if (attempts === 1) { + expect(context).toBeUndefined(); + throw new ResourceOperationPendingError("provider is not ready", { + retryAfterMs: RETRY_AFTER_MS, + callbackContext: { requestId: "request-123" }, + }); + } + expect(context).toEqual({ requestId: "request-123" }); + }, + delete: async () => undefined, + }); + const runtime = createRuntime( + [new PendingResource({ id: "pending" })], + "durable-wait", + { maxOperationAttempts: 3 }, + ); + + await runtime.run("wait-execution"); + expect(attempts).toBe(1); + expect(runtime.scheduler.events).toHaveLength(1); + + await sleep(PAST_RETRY_MS); + await runtime.run("wait-execution"); + expect(attempts).toBe(2); + expect(await runtime.state.get("pending")).toMatchObject({ + id: "pending", + lastOperation: "create", + version: 0, + }); + runtime.close(); + }); + + it("resumes after a crash following the create checkpoint", async () => { + const create = vi.fn(async () => undefined); + const TestResource = resource({ type: "test/durable/resume" }) + .defineSchema({}) + .defineOperations({ create, delete: async () => undefined }); + const runtime = createRuntime( + [new TestResource({ id: "resume" })], + "crash-resume", + { crashAfterStep: "notation:deploy:resume:create:remote:attempt:0" }, + ); + + await expect(runtime.run("resume-execution")).rejects.toThrow( + "simulated process crash", + ); + expect(create).toHaveBeenCalledOnce(); + expect(await runtime.state.get("resume")).toBeUndefined(); + + await runtime.run("resume-execution"); + expect(create).toHaveBeenCalledOnce(); + expect(await runtime.state.get("resume")).toMatchObject({ version: 0 }); + runtime.close(); + }); + + it("resumes after a crash following the delete checkpoint", async () => { + const remove = vi.fn(async () => undefined); + const TestResource = resource({ type: "test/durable/destroy-resume" }) + .defineSchema({}) + .defineOperations({ create: async () => undefined, delete: remove }); + const runtime = createRuntime( + [new TestResource({ id: "destroyed" })], + "destroy-crash-resume", + { crashAfterStep: "notation:destroy:destroyed:delete:remote:attempt:0" }, + ); + + await runtime.run("deploy-before-destroy"); + await expect(runtime.destroy("destroy-execution")).rejects.toThrow( + "simulated process crash", + ); + expect(remove).toHaveBeenCalledOnce(); + expect(await runtime.state.get("destroyed")).toBeDefined(); + + await runtime.destroy("destroy-execution"); + expect(remove).toHaveBeenCalledOnce(); + expect(await runtime.state.get("destroyed")).toBeUndefined(); + runtime.close(); + }); + + it("waits durably for a retryable delete before removing state", async () => { + let attempts = 0; + const PendingDelete = resource({ type: "test/durable/pending-delete" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + delete: async () => { + attempts += 1; + if (attempts === 1) { + throw new ResourceOperationPendingError("delete is not ready", { + retryAfterMs: RETRY_AFTER_MS, + }); + } + }, + }); + const runtime = createRuntime( + [new PendingDelete({ id: "pending-delete" })], + "durable-destroy-wait", + { maxOperationAttempts: 3 }, + ); + + await runtime.run("deploy-before-wait"); + await runtime.destroy("destroy-wait"); + expect(attempts).toBe(1); + expect(await runtime.state.get("pending-delete")).toBeDefined(); + + await sleep(PAST_RETRY_MS); + await runtime.destroy("destroy-wait"); + expect(attempts).toBe(2); + expect(await runtime.state.get("pending-delete")).toBeUndefined(); + runtime.close(); + }); + + it("waits when a resource reports that its post-write read is pending", async () => { + let reads = 0; + const EventuallyReadable = resource({ + type: "test/durable/eventually-readable", + }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + read: async () => { + reads += 1; + if (reads === 1) { + throw new ResourceOperationPendingError( + "resource is not visible yet", + { retryAfterMs: RETRY_AFTER_MS }, + ); + } + return {} as const; + }, + delete: async () => undefined, + }); + const runtime = createRuntime( + [new EventuallyReadable({ id: "eventually-readable" })], + "post-write-read", + { maxOperationAttempts: 3 }, + ); + + await runtime.run("post-write-read-execution"); + expect(reads).toBe(1); + expect(await runtime.state.get("eventually-readable")).toBeUndefined(); + + await sleep(PAST_RETRY_MS); + await runtime.run("post-write-read-execution"); + expect(reads).toBe(2); + expect(await runtime.state.get("eventually-readable")).toMatchObject({ + version: 0, + }); + runtime.close(); + }); + + it("does not infer that not-found after a write is pending", async () => { + const MissingAfterCreate = resource({ + type: "test/durable/missing-after-create", + }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + read: async () => { + throw new ResourceNotFoundError("resource is absent"); + }, + delete: async () => undefined, + }); + const runtime = createRuntime( + [new MissingAfterCreate({ id: "missing-after-create" })], + "missing-after-create", + ); + + await expect(runtime.run("missing-after-create-execution")).rejects.toThrow( + "resource is absent", + ); + expect(await runtime.state.get("missing-after-create")).toBeUndefined(); + runtime.close(); + }); +}); + +describe("dependency ordering", () => { + it("destroys dependents before their dependencies", async () => { + const order: string[] = []; + const Dependency = resource({ type: "test/durable/dependency" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + delete: async () => void order.push("dependency"), + }); + const Dependent = resource({ type: "test/durable/dependent" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + delete: async () => void order.push("dependent"), + }); + const dependency = new Dependency({ id: "dependency" }); + const dependent = new Dependent({ + id: "dependent", + dependencies: { dependency }, + }); + const runtime = createRuntime([dependency, dependent], "destroy-order"); + + await runtime.run("deploy-before-ordered-destroy"); + await runtime.destroy("ordered-destroy"); + + expect(order).toEqual(["dependent", "dependency"]); + runtime.close(); + }); +}); + +describe("conditional state persistence", () => { + it("rejects a state write whose snapshot another writer has moved past", async () => { + const RaceResource = resource({ type: "test/durable/write-race" }) + // Cast: a resource declared without API types constrains every schema + // key to be a key of an `any` API schema, which no named key satisfies. + .defineSchema({ + name: { presence: "required", propertyType: "param" }, + } as any) + .defineOperations({ + create: async () => undefined, + // Moves the store on between the workflow reading its snapshot and + // persisting against it, which is what the conditional write guards. + update: async () => { + await runtime.storeClient.updateStore({ + definition: durable.resourceStateStore, + id: runtime.state.storeId("raced"), + updater: (draft: any) => { + draft.lastOperationAt = "1999-01-01T00:00:00.000Z"; + }, + }); + }, + delete: async () => undefined, + }); + const resources = [ + new RaceResource({ id: "raced", config: { name: "before" } }), + ]; + const runtime = createRuntime(resources, "write-race"); + + await runtime.run("deploy-1"); + resources[0] = new RaceResource({ id: "raced", config: { name: "after" } }); + + await expect(runtime.run("deploy-2")).rejects.toMatchObject({ + name: "VersionConflict", + }); + // The losing write left the other writer's record intact. + expect(await runtime.state.get("raced")).toMatchObject({ + version: 1, + lastOperationAt: "1999-01-01T00:00:00.000Z", + }); + runtime.close(); + }); + + it("rejects a state removal whose snapshot another writer has moved past", async () => { + const RaceResource = resource({ type: "test/durable/delete-race" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + delete: async () => { + await runtime.storeClient.updateStore({ + definition: durable.resourceStateStore, + id: runtime.state.storeId("delete-raced"), + updater: (draft: any) => { + draft.lastOperationAt = "1999-01-01T00:00:00.000Z"; + }, + }); + }, + }); + const runtime = createRuntime( + [new RaceResource({ id: "delete-raced" })], + "delete-race", + ); + + await runtime.run("deploy-1"); + await expect(runtime.destroy("destroy-1")).rejects.toMatchObject({ + name: "VersionConflict", + // A conflict names the version the record moved to; only a genuinely + // absent record may be reported as "missing". + message: expect.stringMatching(/expected 0, got 1$/), + }); + // State survives a removal that could not be proven safe. + expect(await runtime.state.get("delete-raced")).toBeDefined(); + runtime.close(); + }); +}); + +describe("deployment hold", () => { + it("serializes concurrent deployments through durable store waiting", async () => { + let unblockCreate!: () => void; + const blocked = new Promise((resolve) => { + unblockCreate = resolve; + }); + let started!: () => void; + const createStarted = new Promise((resolve) => { + started = resolve; + }); + const create = vi.fn(async () => { + started(); + await blocked; + }); + const TestResource = resource({ type: "test/durable/concurrent" }) + .defineSchema({}) + .defineOperations({ create, delete: async () => undefined }); + const runtime = createRuntime( + [new TestResource({ id: "shared" })], + "concurrent", + ); + + const first = runtime.run("deployment-a"); + await createStarted; + await runtime.run("deployment-b"); + expect(create).toHaveBeenCalledOnce(); + + unblockCreate(); + await first; + const wake = runtime.scheduler.events.find( + (event) => event.executionId === "deployment-b", + ); + expect(wake).toBeDefined(); + await runtime.runner.run(wake!, logger); + + expect(create).toHaveBeenCalledOnce(); + expect(await runtime.state.values()).toHaveLength(1); + runtime.close(); + }); + + it("still holds the deployment when a failed execution is resumed", async () => { + // The failure has to live in plain generator code. A step that fails + // caches a StepError, and a cached StepError is rethrown on replay before + // the step's function is reached, so no later work would ever run + // uncached. For a resource with persisted state, decideAction calls + // toComparable outside any step, after the resource's reads have been + // checkpointed. + let failOnSecond = true; + const holders: Array = []; + const Resource = resource({ type: "test/durable/hold-replay" }) + .defineSchema({}) + .defineOperations({ + create: async () => { + const snapshot = await runtime.storeClient.getStore({ + definition: durable.deploymentHoldStore, + id: "hold-replay", + }); + holders.push(snapshot.state.holder); + }, + delete: async () => undefined, + }); + + const first = new Resource({ id: "first" }); + const second = new Resource({ id: "second" }); + const third = new Resource({ id: "third" }); + const toComparable = second.toComparable.bind(second); + second.toComparable = (output) => { + if (failOnSecond) throw new Error("simulated mid-deployment failure"); + return toComparable(output); + }; + const runtime = createRuntime([first, second, third], "hold-replay"); + // Persisted state for `second`, so deciding its action reaches the + // failure seam in toComparable. + await seedResourceState( + runtime.storeClient, + runtime.state.storeId("second"), + "second", + ); + + await expect(runtime.run("replayed-execution")).rejects.toThrow( + "simulated mid-deployment failure", + ); + expect(holders).toEqual(["replayed-execution"]); + + failOnSecond = false; + await runtime.run("replayed-execution"); + + // The third resource's create is the first uncached work after the + // failure, so it observes whichever hold the resumed execution is running + // under. Releasing on the way out of a failure would leave it null here. + expect(holders).toEqual(["replayed-execution", "replayed-execution"]); + runtime.close(); + }); + + it("emits a hold waiting event when another execution holds the deployment", async () => { + let unblockCreate!: () => void; + const blocked = new Promise((resolve) => { + unblockCreate = resolve; + }); + let started!: () => void; + const createStarted = new Promise((resolve) => { + started = resolve; + }); + const TestResource = resource({ type: "test/durable/hold" }) + .defineSchema({}) + .defineOperations({ + create: async () => { + started(); + await blocked; + }, + delete: async () => undefined, + }); + const events: ReconcilerEvent[] = []; + const runtime = createRuntime( + [new TestResource({ id: "held" })], + "hold-waiting", + { emit: (event) => void events.push(event) }, + ); + + const first = runtime.run("holder-execution"); + await createStarted; + await runtime.run("waiter-execution"); + + expect( + events.find((event) => event.event === "reconciler.hold.waiting"), + ).toMatchObject({ + level: "warn", + deploymentId: "hold-waiting", + executionId: "waiter-execution", + holderExecutionId: "holder-execution", + }); + + unblockCreate(); + await first; + runtime.close(); + }); +}); + +describe("deployment hold clearing", () => { + it("clears a hold its named holder still has, and unblocks the deployment", async () => { + const create = vi.fn(async () => undefined); + const Resource = resource({ type: "test/durable/takeover" }) + .defineSchema({}) + .defineOperations({ create, delete: async () => undefined }); + const runtime = createRuntime([new Resource({ id: "held" })], "takeover"); + await runtime.storeClient.getOrCreateStore({ + definition: durable.deploymentHoldStore, + id: "takeover", + initial: { holder: "abandoned-execution" }, + }); + + const result = await durable.clearDeploymentHold({ + storeClient: runtime.storeClient, + deploymentId: "takeover", + fromExecutionId: "abandoned-execution", + }); + + expect(result).toEqual({ + cleared: true, + previousHolder: "abandoned-execution", + }); + await runtime.run("later-execution"); + expect(create).toHaveBeenCalledOnce(); + runtime.close(); + }); + + it("refuses to clear a hold that has moved to another execution", async () => { + const runtime = createRuntime([], "takeover-race"); + await runtime.storeClient.getOrCreateStore({ + definition: durable.deploymentHoldStore, + id: "takeover-race", + initial: { holder: "current-execution" }, + }); + + const result = await durable.clearDeploymentHold({ + storeClient: runtime.storeClient, + deploymentId: "takeover-race", + fromExecutionId: "abandoned-execution", + }); + + expect(result).toEqual({ cleared: false, holder: "current-execution" }); + const snapshot = await runtime.storeClient.getStore({ + definition: durable.deploymentHoldStore, + id: "takeover-race", + }); + expect(snapshot.state.holder).toBe("current-execution"); + runtime.close(); + }); +}); + +describe("deployment scoping", () => { + it("scopes store listing to the exact deployment despite prefix-like IDs", async () => { + const database = createSqliteDb({ path: ":memory:" }); + const storeClient = new SqliteStoreClient({ + db: database, + schedulerClient: new TestScheduler(), + }); + const app = new durable.DurableStateBackend(storeClient, "app"); + const appBlue = new durable.DurableStateBackend(storeClient, "app:blue"); + + await seedResourceState(storeClient, app.storeId("site"), "site"); + await seedResourceState(storeClient, appBlue.storeId("site"), "blue-site"); + + // A deployment named "app:blue" falls inside a naive "app:" prefix scan; + // encoding the deployment id is what keeps the two listings disjoint. + expect((await app.values()).map((node) => node.id)).toEqual(["site"]); + expect((await appBlue.values()).map((node) => node.id)).toEqual([ + "blue-site", + ]); + expect(await app.get("site")).toMatchObject({ id: "site" }); + expect(await appBlue.get("site")).toMatchObject({ id: "blue-site" }); + database.close(); + }); +}); + +describe("orphan deletion", () => { + it("deletes orphaned resources through the registry on a later deployment", async () => { + const deleteSpy = vi.fn(async () => undefined); + const OrphanResource = resource({ type: "test/durable/orphan" }) + .defineSchema({}) + .defineOperations({ create: async () => undefined, delete: deleteSpy }); + const resources: BaseResource[] = [new OrphanResource({ id: "orphan" })]; + const runtime = createRuntime(resources, "orphan-deletion", { + registry: createResourceRegistry([OrphanResource]), + }); + + await runtime.run("deploy-1"); + expect(await runtime.state.values()).toHaveLength(1); + + resources.length = 0; + await runtime.run("deploy-2"); + + expect(deleteSpy).toHaveBeenCalledOnce(); + expect(await runtime.state.values()).toHaveLength(0); + expect(await runtime.state.get("orphan")).toBeUndefined(); + runtime.close(); + }); +}); + +describe("drift detection and repair", () => { + it("emits drift detection and repairs remote drift with update", async () => { + let remote = { name: "expected" }; + const updateSpy = vi.fn(async () => { + remote = { name: "expected" }; + }); + const DriftResource = resource({ type: "test/durable/drift" }) + // Cast: a resource declared without API types constrains every schema + // key to be a key of an `any` API schema, which no named key satisfies. + .defineSchema({ + name: { presence: "required", propertyType: "param" }, + } as any) + .defineOperations({ + create: (async () => remote) as any, + read: async () => remote, + update: updateSpy, + delete: async () => undefined, + }); + const events: ReconcilerEvent[] = []; + const runtime = createRuntime( + [new DriftResource({ id: "drifted", config: { name: "expected" } })], + "drift-repair", + { driftDetection: true, emit: (event) => void events.push(event) }, + ); + + await runtime.run("deploy-1"); + remote = { name: "drifted" }; + await runtime.run("deploy-2"); + + expect(updateSpy).toHaveBeenCalledOnce(); + expect( + events.find((event) => event.event === "reconciler.drift.detected"), + ).toMatchObject({ resourceId: "drifted", diff: { name: "expected" } }); + expect( + events.filter( + (event) => + event.event === "reconciler.deploy.decision" && + event.decision === "drift-update", + ), + ).toHaveLength(1); + runtime.close(); + }); + + it("trusts a noop for a resource with no read instead of reading during drift detection", async () => { + const ReadlessResource = resource({ type: "test/durable/readless" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + update: async () => undefined, + delete: async () => undefined, + }); + const events: ReconcilerEvent[] = []; + const runtime = createRuntime( + [new ReadlessResource({ id: "readless" })], + "readless-drift", + { driftDetection: true, emit: (event) => void events.push(event) }, + ); + + await runtime.run("deploy-1"); + events.length = 0; + await runtime.run("deploy-2"); + + // With nothing to read, the drift read would only replay persisted output + // through read skip/success lifecycle events a plan never emits. + expect( + events.filter( + (event) => + event.event === "reconciler.operation.lifecycle" && + event.operation === "read", + ), + ).toEqual([]); + expect( + events.find((event) => event.event === "reconciler.deploy.decision"), + ).toMatchObject({ resourceId: "readless", decision: "noop" }); + runtime.close(); + }); + + it("reads the remote during a dry run rather than reporting false drift", async () => { + const remote = { name: "expected" }; + const read = vi.fn(async () => remote); + const DryRunDriftResource = resource({ type: "test/durable/dry-run-drift" }) + .defineSchema({ + name: { presence: "required", propertyType: "param" }, + } as any) + .defineOperations({ + create: (async () => remote) as any, + read, + update: async () => undefined, + delete: async () => undefined, + }); + const events: ReconcilerEvent[] = []; + const options = { + driftDetection: true, + dryRun: false, + emit: (event: ReconcilerEvent) => void events.push(event), + }; + const runtime = createRuntime( + [new DryRunDriftResource({ id: "steady", config: { name: "expected" } })], + "dry-run-drift", + options, + ); + + await runtime.run("deploy-1"); + const readsAfterDeploy = read.mock.calls.length; + + options.dryRun = true; + events.length = 0; + await runtime.run("dry-run"); + + // Suppressing the drift read under dryRun would leave decideAction + // diffing an empty read against the desired params, which reports every + // param as drift; skipping it entirely would make a dry run unable to + // report the drift it exists to report. + expect(read.mock.calls.length).toBeGreaterThan(readsAfterDeploy); + expect( + events.filter((event) => event.event === "reconciler.drift.detected"), + ).toEqual([]); + expect( + events.find((event) => event.event === "reconciler.deploy.decision"), + ).toMatchObject({ resourceId: "steady", decision: "noop" }); + runtime.close(); + }); +}); + +function createRuntime( + resources: BaseResource[], + deploymentId: string, + options: { + maxOperationAttempts?: number; + crashAfterStep?: string; + registry?: ResourceRegistry; + driftDetection?: boolean; + dryRun?: boolean; + emit?: (event: ReconcilerEvent) => void; + } = {}, +) { + const database = createSqliteDb({ path: ":memory:" }); + const scheduler = new TestScheduler(); + const sqliteHeap = new SqliteHeapClient(database); + const heap = options.crashAfterStep + ? new CrashAfterWriteHeap(sqliteHeap, options.crashAfterStep) + : sqliteHeap; + const storeClient = new SqliteStoreClient({ + db: database, + schedulerClient: scheduler, + }); + const state = new durable.DurableStateBackend(storeClient, deploymentId); + const deploy = workflow(async function* (step, event) { + yield* durable.deploy(step, { + executionId: event.executionId, + resources, + state, + registry: options.registry, + driftDetection: options.driftDetection ?? false, + // Read at execution time, so a test can switch it between runs. + dryRun: options.dryRun, + emit: options.emit, + maxOperationAttempts: options.maxOperationAttempts, + }); + }); + const destroy = workflow(async function* (step, event) { + yield* durable.destroy(step, { + executionId: event.executionId, + resources, + state, + registry: options.registry, + emit: options.emit, + maxOperationAttempts: options.maxOperationAttempts, + }); + }); + const router = createWorkflowRouter({ deploy, destroy }); + const runner = new WorkflowRunner({ + router, + heapClient: heap, + storeClient, + schedulerClient: scheduler, + logger, + }); + + return { + runner, + scheduler, + state, + storeClient, + run(executionId: string) { + return runner.run( + { + workflowId: "deploy", + executionId, + params: {}, + context: new Map(), + }, + logger, + ); + }, + destroy(executionId: string) { + return runner.run( + { + workflowId: "destroy", + executionId, + params: {}, + context: new Map(), + }, + logger, + ); + }, + close() { + database.close(); + }, + }; +} + +class TestScheduler { + readonly events: WorkflowEvent[] = []; + + async requestWakeUp(event: WorkflowEvent) { + this.events.push(event); + } +} + +class CrashAfterWriteHeap implements HeapClient { + #crashed = false; + + constructor( + private readonly inner: HeapClient, + private readonly crashAfterStep: string, + ) {} + + readStep(params: { executionId: string; stepKey: string }) { + return this.inner.readStep(params); + } + + async writeStep(params: { + executionId: string; + stepKey: string; + stepAttempt: number; + stepDone: boolean; + stepResponseJson: string; + }) { + await this.inner.writeStep(params); + if ( + !this.#crashed && + params.stepKey === this.crashAfterStep && + params.stepDone + ) { + this.#crashed = true; + throw new Error("simulated process crash"); + } + } +} + +function seedResourceState( + storeClient: SqliteStoreClient, + storeId: string, + resourceId: string, +) { + return storeClient.getOrCreateStore({ + definition: durable.resourceStateStore, + id: storeId, + initial: resourceStateRecord(resourceId), + }); +} + +function resourceStateRecord(id: string) { + return { + id, + type: "test/durable/state", + groupId: -1, + groupType: "", + config: {}, + params: {}, + output: {}, + lastOperation: "create" as const, + lastOperationAt: new Date().toISOString(), + }; +} diff --git a/packages/reconciler/test/logger-subscriber.test.ts b/packages/reconciler/test/logger-subscriber.test.ts index 49569dd..0614d5b 100644 --- a/packages/reconciler/test/logger-subscriber.test.ts +++ b/packages/reconciler/test/logger-subscriber.test.ts @@ -27,6 +27,13 @@ describe("logger reconciler subscriber", () => { resourceId: "resource-2", resourceType: "test/service/subscriber", }); + await emit({ + level: "warn", + event: "reconciler.hold.waiting", + deploymentId: "deployment-1", + executionId: "execution-2", + holderExecutionId: "execution-1", + }); await emit({ level: "error", event: "reconciler.operation.lifecycle", @@ -39,7 +46,12 @@ describe("logger reconciler subscriber", () => { }); expect(info).toHaveBeenCalledOnce(); - expect(warn).toHaveBeenCalledOnce(); + expect(warn).toHaveBeenCalledTimes(2); + expect(warn).toHaveBeenNthCalledWith( + 2, + "reconciler.hold.waiting", + expect.objectContaining({ level: "warn" }), + ); expect(error).toHaveBeenCalledOnce(); }); }); diff --git a/packages/reconciler/test/operation.workflows.test.ts b/packages/reconciler/test/operation.workflows.test.ts index 2afc6df..fafb595 100644 --- a/packages/reconciler/test/operation.workflows.test.ts +++ b/packages/reconciler/test/operation.workflows.test.ts @@ -11,54 +11,42 @@ import { type OperationLifecycleEvent, type StepRunner, } from "../src/operations"; +import { toEmitStep } from "../src/events"; +import { runOperation } from "../src/step-runner"; -function createStepRunnerDouble(): StepRunner { +function createStepRunnerDouble() { const run = vi.fn(async function* ( - arg1: string | (() => T | Promise), - arg2?: () => T | Promise, + _key: string, + fn: () => T | Promise, ): AsyncGenerator { - const fn = (typeof arg1 === "string" ? arg2 : arg1) as () => T | Promise; - if (!fn) { - throw new Error("Missing run function"); - } - return await fn(); }); - const delay = vi.fn(async function* (): AsyncGenerator< - unknown, - void, - unknown - > { + const delay = vi.fn(async function* ( + _key: string, + _ms: number, + ): AsyncGenerator { return; }); - return { - run, - delay, + // vi.fn erases the generic, so the seam's signature is restored here. + const runner: StepRunner = { + run: run as unknown as StepRunner["run"], + delay: delay as unknown as StepRunner["delay"], + scope: () => runner, }; -} -async function runOperation(operation: AsyncGenerator) { - let next = await operation.next(); - while (!next.done) { - next = await operation.next(); - } - return next.value; + return runner; } describe("operation workflows", () => { it("create performs create + read-after-create + state persistence", async () => { const step = createStepRunnerDouble(); const events: OperationLifecycleEvent[] = []; - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; + const persist = vi.fn(async function* () {}); let createAttempts = 0; - const createMock = vi.fn(async (_params, context) => { + const createMock = vi.fn(async (_params: unknown, context: unknown) => { createAttempts += 1; if (createAttempts === 1) { expect(context).toBeUndefined(); @@ -75,7 +63,8 @@ describe("operation workflows", () => { const TestResource = resource({ type: "test/service/create" }) .defineSchema({}) .defineOperations({ - create: createMock, + // Cast: an empty schema declares no primary key to return. + create: createMock as any, read: async () => ({ remoteId: "abc", status: "ready" }), delete: async () => undefined, }); @@ -85,16 +74,26 @@ describe("operation workflows", () => { await runOperation( createResourceOperation(step, { resource: testResource, - state, - expectedRev: 0, - emit: async (event) => { - events.push(event); - }, + resourceParams: await testResource.getParams(), + persist, + emit: toEmitStep((event) => void events.push(event)), }), ); expect(createAttempts).toBe(2); - expect(state.update).toHaveBeenCalledOnce(); + expect(persist).toHaveBeenCalledOnce(); + expect(persist).toHaveBeenCalledWith({ + id: "test-create", + groupId: testResource.groupId, + groupType: testResource.groupType, + type: TestResource.type, + lastOperation: "create", + lastOperationAt: expect.any(String), + config: testResource.config, + params: {}, + // The resource declares no schema, so nothing survives toState. + output: {}, + }); expect(createMock).toHaveBeenNthCalledWith( 1, await testResource.getParams(), @@ -118,17 +117,12 @@ describe("operation workflows", () => { it("read follows pending retry instructions", async () => { const step = createStepRunnerDouble(); - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; let readAttempts = 0; const TestResource = resource({ type: "test/service/read" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, read: async (_key, context) => { readAttempts += 1; if (readAttempts < 3) { @@ -148,7 +142,8 @@ describe("operation workflows", () => { const result = await runOperation( readResourceOperation(step, { resource: testResource, - state, + resourceParams: await testResource.getParams(), + emit: toEmitStep(), }), ); @@ -168,11 +163,6 @@ describe("operation workflows", () => { it("fails when an operation remains pending past the safety limit", async () => { const step = createStepRunnerDouble(); - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; const read = vi.fn(async () => { throw new ResourceOperationPendingError("still pending", { retryAfterMs: 10, @@ -181,7 +171,7 @@ describe("operation workflows", () => { const TestResource = resource({ type: "test/service/pending-limit" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, read, delete: async () => undefined, }); @@ -190,7 +180,8 @@ describe("operation workflows", () => { runOperation( readResourceOperation(step, { resource: new TestResource({ id: "pending-limit" }), - state, + resourceParams: {}, + emit: toEmitStep(), maxOperationAttempts: 2, }), ), @@ -201,15 +192,11 @@ describe("operation workflows", () => { it("does not infer that not-found after creation is retryable", async () => { const step = createStepRunnerDouble(); - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; + const persist = vi.fn(async function* () {}); const TestResource = resource({ type: "test/service/eventually-visible" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, read: async () => { throw new ResourceNotFoundError("resource is absent"); }, @@ -220,27 +207,24 @@ describe("operation workflows", () => { runOperation( createResourceOperation(step, { resource: new TestResource({ id: "eventually-visible" }), - state, - expectedRev: 0, + resourceParams: {}, + persist, + emit: toEmitStep(), }), ), ).rejects.toThrowError("resource is absent"); - expect(state.update).not.toHaveBeenCalled(); + expect(persist).not.toHaveBeenCalled(); }); it("delete treats an already-absent remote as success through its idempotent resource contract", async () => { const step = createStepRunnerDouble(); const events: OperationLifecycleEvent[] = []; - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; + const remove = vi.fn(async function* () {}); const TestResource = resource({ type: "test/service/delete" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, delete: async () => undefined, }); @@ -249,30 +233,56 @@ describe("operation workflows", () => { await runOperation( deleteResourceOperation(step, { resource: testResource, - state, - expectedRev: 1, - emit: async (event) => { - events.push(event); + remove, + emit: toEmitStep((event) => void events.push(event)), + }), + ); + + // State is removed only after the provider delete resolves; which record + // and version that targets is the driver's concern, not the operation's. + expect(remove).toHaveBeenCalledOnce(); + expect(events.map((event) => event.status)).toEqual(["start", "success"]); + }); + + it("delete treats ResourceNotFoundError as the resource already being absent", async () => { + const step = createStepRunnerDouble(); + const events: OperationLifecycleEvent[] = []; + const remove = vi.fn(async function* () {}); + + const TestResource = resource({ type: "test/service/delete-absent" }) + .defineSchema({}) + .defineOperations({ + create: (async () => ({})) as any, + delete: async () => { + throw new ResourceNotFoundError("resource is already gone"); }, + }); + + const testResource = new TestResource({ id: "test-delete-absent" }); + + await runOperation( + deleteResourceOperation(step, { + resource: testResource, + remove, + emit: toEmitStep((event) => void events.push(event)), }), ); - expect(state.delete).toHaveBeenCalledWith("test-delete", 1); + // Absence is delete's goal state: state is removed and the operation + // reports success, which is what makes a crash-window replayed delete + // idempotent even when the handler surfaces the provider's missing error. + expect(remove).toHaveBeenCalledOnce(); expect(events.map((event) => event.status)).toEqual(["start", "success"]); }); it("delete rethrows an unclassified resource error", async () => { const step = createStepRunnerDouble(); - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; + const remove = vi.fn(async function* () {}); const TestResource = resource({ type: "test/service/delete-miss" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, delete: async () => { const err = new Error("still exists"); err.name = "DifferentError"; @@ -286,8 +296,8 @@ describe("operation workflows", () => { runOperation( deleteResourceOperation(step, { resource: testResource, - state, - expectedRev: 1, + remove, + emit: toEmitStep(), }), ), ).rejects.toMatchObject({ @@ -295,17 +305,13 @@ describe("operation workflows", () => { message: "still exists", }); - expect(state.delete).not.toHaveBeenCalled(); + expect(remove).not.toHaveBeenCalled(); }); it("emits structured error details on operation failure", async () => { const step = createStepRunnerDouble(); const events: OperationLifecycleEvent[] = []; - const state = { - get: vi.fn(async () => undefined), - update: vi.fn(async () => undefined), - delete: vi.fn(async () => undefined), - }; + const persist = vi.fn(async function* () {}); const TestResource = resource({ type: "test/service/create-error" }) .defineSchema({}) @@ -324,11 +330,9 @@ describe("operation workflows", () => { runOperation( createResourceOperation(step, { resource: testResource, - state, - expectedRev: 0, - emit: async (event) => { - events.push(event); - }, + resourceParams: {}, + persist, + emit: toEmitStep((event) => void events.push(event)), }), ), ).rejects.toMatchObject({ name: "CreateFailed", message: "boom" }); diff --git a/packages/reconciler/test/planner.test.ts b/packages/reconciler/test/planner.test.ts new file mode 100644 index 0000000..b4b0c05 --- /dev/null +++ b/packages/reconciler/test/planner.test.ts @@ -0,0 +1,150 @@ +import { + ResourceNotFoundError, + ResourceOperationPendingError, + resource, +} from "@notation/resource"; +import { MemoryStateBackend } from "@notation/state"; +import { describe, expect, it } from "vitest"; +import { createPlan } from "../src/planner"; + +describe("createPlan", () => { + it("plans desired creates and persisted orphans without mutation execution", async () => { + const TestResource = resource({ type: "test/planner/resource" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + delete: async () => undefined, + }); + const state = new MemoryStateBackend({ + orphan: { + version: 1, + id: "orphan", + type: TestResource.type, + config: {}, + params: {}, + output: {}, + lastOperation: "create", + lastOperationAt: "2026-07-22T00:00:00.000Z", + }, + }); + + const plan = await createPlan({ + resources: [new TestResource({ id: "desired" })], + state, + driftDetection: false, + }); + + expect(plan.nodes).toEqual([ + expect.objectContaining({ id: "desired", decision: "create" }), + expect.objectContaining({ id: "orphan", decision: "delete-orphan" }), + ]); + }); + + it("propagates unexpected read failures", async () => { + const TestResource = resource({ type: "test/planner/read-failure" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + read: async () => { + throw new Error("access denied"); + }, + delete: async () => undefined, + }); + const state = new MemoryStateBackend({ + existing: { + version: 1, + id: "existing", + type: TestResource.type, + config: {}, + params: {}, + output: {}, + lastOperation: "create", + lastOperationAt: "2026-07-22T00:00:00.000Z", + }, + }); + + await expect( + createPlan({ + resources: [new TestResource({ id: "existing" })], + state, + }), + ).rejects.toThrow("access denied"); + }); + + it("plans recreation when the resource reports absence", async () => { + const TestResource = resource({ type: "test/planner/absent" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + read: async () => { + throw new ResourceNotFoundError("resource is absent"); + }, + delete: async () => undefined, + }); + const state = new MemoryStateBackend({ + existing: { + version: 1, + id: "existing", + type: TestResource.type, + config: {}, + params: {}, + output: {}, + lastOperation: "create", + lastOperationAt: "2026-07-22T00:00:00.000Z", + }, + }); + + const plan = await createPlan({ + resources: [new TestResource({ id: "existing" })], + state, + }); + + expect(plan.nodes[0]).toMatchObject({ + id: "existing", + decision: "drift-recreate", + }); + }); + + it("waits for a pending read before planning", async () => { + let attempts = 0; + const TestResource = resource({ type: "test/planner/pending" }) + .defineSchema({}) + .defineOperations({ + create: async () => undefined, + read: async () => { + attempts += 1; + if (attempts === 1) { + throw new ResourceOperationPendingError( + "Waiting for the provider", + { retryAfterMs: 0 }, + ); + } + return {}; + }, + delete: async () => undefined, + }); + const state = new MemoryStateBackend({ + existing: { + version: 1, + id: "existing", + type: TestResource.type, + config: {}, + params: {}, + output: {}, + lastOperation: "create", + lastOperationAt: "2026-07-22T00:00:00.000Z", + }, + }); + + const plan = await createPlan({ + resources: [new TestResource({ id: "existing" })], + state, + }); + + expect(plan.nodes[0]).toMatchObject({ + id: "existing", + decision: "noop", + }); + expect(attempts).toBe(2); + }); +}); diff --git a/packages/reconciler/test/reconciler.deploy.test.ts b/packages/reconciler/test/reconciler.deploy.test.ts deleted file mode 100644 index eaa5ce5..0000000 --- a/packages/reconciler/test/reconciler.deploy.test.ts +++ /dev/null @@ -1,807 +0,0 @@ -import { describe, expect, it, vi } from "vitest"; -import { resource, ResourceNotFoundError } from "@notation/resource"; -import { - LeaseConflict, - MemoryStateBackend, - RevConflict, - type StateNode, -} from "@notation/state"; -import { Reconciler, createResourceRegistry } from "../src"; - -function createMemoryState(initial: Record = {}) { - const store: Record = { ...initial }; - - return { - store, - get: vi.fn(async (id: string) => store[id]), - update: vi.fn( - async (id: string, expectedRev: number, patch: Partial) => { - const actualRev = store[id]?.rev ?? 0; - if (actualRev !== expectedRev) { - throw new RevConflict(id, expectedRev, store[id]?.rev); - } - const rev = actualRev + 1; - store[id] = { - ...(store[id] ?? {}), - ...patch, - rev, - } as StateNode; - return { rev }; - }, - ), - delete: vi.fn(async (id: string, expectedRev: number) => { - const actualRev = store[id]?.rev ?? 0; - if (actualRev !== expectedRev) { - throw new RevConflict(id, expectedRev, store[id]?.rev); - } - delete store[id]; - }), - values: vi.fn(async () => Object.values(store)), - lease: vi.fn(async (scope: string, ttl: number) => { - let expiresAt = new Date(Date.now() + ttl).toISOString(); - return { - scope, - get expiresAt() { - return expiresAt; - }, - renew: vi.fn(async (nextTtl: number) => { - expiresAt = new Date(Date.now() + nextTtl).toISOString(); - return expiresAt; - }), - release: vi.fn(async () => undefined), - }; - }), - }; -} - -function createTestResourceClass(opts: { - type: `${string}/${string}/${string}`; - create?: ( - params: Record, - ) => Promise | void>; - read?: (key: Record) => Promise>; - update?: ( - key: Record, - patch: Record, - params: Record, - state: Record, - ) => Promise; - delete?: ( - key: Record, - state: Record, - ) => Promise; -}) { - return resource({ type: opts.type }) - .defineSchema({ - name: { - presence: "required", - propertyType: "param", - valueType: "string" as any, - }, - }) - .defineOperations({ - create: opts.create ?? (async () => ({})), - read: opts.read, - update: opts.update, - delete: opts.delete ?? (async () => undefined), - }); -} - -const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); -const found = (output: Record) => output; - -describe("reconciler deploy", () => { - it("chooses create vs update from desired params vs state", async () => { - const createSpy = vi.fn(async () => ({ name: "new" })); - const updateSpy = vi.fn(async () => undefined); - - const CreateResource = createTestResourceClass({ - type: "test/service/create-choice", - create: createSpy, - read: async () => found({ name: "new" }), - }); - const UpdateResource = createTestResourceClass({ - type: "test/service/update-choice", - update: updateSpy, - read: async () => found({ name: "new" }), - }); - - const state = createMemoryState({ - existing: { - rev: 1, - id: "existing", - groupId: -1, - groupType: "", - type: UpdateResource.type, - config: { name: "old" }, - params: { name: "old" }, - output: { name: "old" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const events: string[] = []; - const reconciler = new Reconciler({ - state, - driftDetection: false, - emit: async (event) => { - if ("operation" in event) { - events.push(`${event.operation}:${event.status}:${event.resourceId}`); - } - }, - }); - - await reconciler.deploy([ - new CreateResource({ id: "new", config: { name: "new" } }), - new UpdateResource({ id: "existing", config: { name: "new" } }), - ]); - - expect(createSpy).toHaveBeenCalledOnce(); - expect(updateSpy).toHaveBeenCalledOnce(); - expect(updateSpy.mock.calls[0]?.[1]).toEqual({ name: "new" }); - expect(events).toContain("create:success:new"); - expect(events).toContain("update:success:existing"); - }); - - it("persists first-time creates with an expect-absent revision", async () => { - const CreateResource = createTestResourceClass({ - type: "test/service/first-create", - create: async () => ({ name: "new" }), - read: async () => found({ name: "new" }), - }); - const state = createMemoryState(); - const reconciler = new Reconciler({ state, driftDetection: false }); - - await reconciler.deploy([ - new CreateResource({ id: "new", config: { name: "new" } }), - ]); - - expect(state.update).toHaveBeenCalledWith("new", 0, expect.any(Object)); - }); - - it("leases a resource before remote create so concurrent deploys cannot duplicate it", async () => { - let signalCreateStarted!: () => void; - const createStarted = new Promise((resolve) => { - signalCreateStarted = resolve; - }); - let allowCreateToFinish!: () => void; - const createCanFinish = new Promise((resolve) => { - allowCreateToFinish = resolve; - }); - const createSpy = vi.fn(async () => { - signalCreateStarted(); - await createCanFinish; - return { name: "new" }; - }); - const CreateResource = createTestResourceClass({ - type: "test/service/concurrent-create", - create: createSpy, - read: async () => found({ name: "new" }), - }); - const state = new MemoryStateBackend(); - const first = new Reconciler({ state, driftDetection: false }); - const second = new Reconciler({ state, driftDetection: false }); - - const firstDeploy = first.deploy([ - new CreateResource({ id: "new", config: { name: "new" } }), - ]); - await createStarted; - - await expect( - second.deploy([ - new CreateResource({ id: "new", config: { name: "new" } }), - ]), - ).rejects.toBeInstanceOf(LeaseConflict); - - allowCreateToFinish(); - await firstDeploy; - expect(createSpy).toHaveBeenCalledOnce(); - }); - - it("reads remote state after an update conflict instead of repeating the update", async () => { - let remoteName = "old"; - const readSpy = vi.fn(async () => found({ name: remoteName })); - const updateSpy = vi.fn(async (_key, _patch, params) => { - remoteName = params.name as string; - }); - const UpdateResource = createTestResourceClass({ - type: "test/service/update-conflict", - read: readSpy, - update: updateSpy, - }); - const state = createMemoryState({ - existing: { - rev: 1, - id: "existing", - groupId: -1, - groupType: "", - type: UpdateResource.type, - config: { name: "old" }, - params: { name: "old" }, - output: { name: "old" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - const updateState = state.update.getMockImplementation()!; - state.update - .mockImplementationOnce(async () => { - state.store.existing = { - ...state.store.existing!, - rev: 2, - config: { name: "concurrent" }, - params: { name: "concurrent" }, - output: { name: "concurrent" }, - }; - throw new RevConflict("existing", 1, 2); - }) - .mockImplementation(updateState); - - const reconciler = new Reconciler({ state, driftDetection: false }); - await reconciler.deploy([ - new UpdateResource({ id: "existing", config: { name: "new" } }), - ]); - - expect(updateSpy).toHaveBeenCalledOnce(); - expect(readSpy).toHaveBeenCalledTimes(2); - expect(state.update).toHaveBeenLastCalledWith( - "existing", - 2, - expect.objectContaining({ - params: { name: "new" }, - output: { name: "new" }, - lastOperation: "drift", - }), - ); - expect(state.store.existing).toMatchObject({ - rev: 3, - params: { name: "new" }, - output: { name: "new" }, - }); - }); - - it("reads remote state after a create conflict instead of creating twice", async () => { - let remoteName: string | undefined; - const createSpy = vi.fn(async (params) => { - remoteName = params.name as string; - return { name: remoteName }; - }); - const readSpy = vi.fn(async () => found({ name: remoteName! })); - const CreateResource = createTestResourceClass({ - type: "test/service/create-conflict", - create: createSpy, - read: readSpy, - }); - const state = createMemoryState(); - const updateState = state.update.getMockImplementation()!; - state.update - .mockImplementationOnce(async () => { - state.store.new = { - rev: 1, - id: "new", - groupId: -1, - groupType: "", - type: CreateResource.type, - config: { name: "concurrent" }, - params: { name: "concurrent" }, - output: { name: "concurrent" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }; - throw new RevConflict("new", 0, 1); - }) - .mockImplementation(updateState); - - const reconciler = new Reconciler({ state, driftDetection: false }); - await reconciler.deploy([ - new CreateResource({ id: "new", config: { name: "new" } }), - ]); - - expect(createSpy).toHaveBeenCalledOnce(); - expect(readSpy).toHaveBeenCalledTimes(2); - expect(state.store.new).toMatchObject({ - rev: 2, - params: { name: "new" }, - output: { name: "new" }, - lastOperation: "drift", - }); - }); - - it("does not blindly retry a conflicted mutation without a read operation", async () => { - const updateSpy = vi.fn(async () => undefined); - const UpdateResource = createTestResourceClass({ - type: "test/service/unreadable-conflict", - update: updateSpy, - }); - const state = createMemoryState({ - existing: { - rev: 1, - id: "existing", - groupId: -1, - groupType: "", - type: UpdateResource.type, - config: { name: "old" }, - params: { name: "old" }, - output: { name: "old" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - state.update.mockImplementationOnce(async () => { - state.store.existing = { ...state.store.existing!, rev: 2 }; - throw new RevConflict("existing", 1, 2); - }); - - const reconciler = new Reconciler({ state, driftDetection: false }); - await expect( - reconciler.deploy([ - new UpdateResource({ id: "existing", config: { name: "new" } }), - ]), - ).rejects.toMatchObject({ - id: "existing", - expectedRev: 1, - actualRev: 2, - }); - - expect(updateSpy).toHaveBeenCalledOnce(); - expect(state.update).toHaveBeenCalledOnce(); - }); - - it("runs independent resources concurrently per dependency depth", async () => { - const marks: Record = {}; - - const AResource = createTestResourceClass({ - type: "test/service/a", - create: async () => { - marks.aStart = Date.now(); - await sleep(60); - marks.aEnd = Date.now(); - return { name: "a" }; - }, - read: async () => found({ name: "a" }), - }); - const CResource = createTestResourceClass({ - type: "test/service/c", - create: async () => { - marks.cStart = Date.now(); - await sleep(60); - marks.cEnd = Date.now(); - return { name: "c" }; - }, - read: async () => found({ name: "c" }), - }); - const BResource = createTestResourceClass({ - type: "test/service/b", - create: async () => { - marks.bStart = Date.now(); - return { name: "b" }; - }, - read: async () => found({ name: "b" }), - }); - - const state = createMemoryState(); - const resourceA = new AResource({ id: "a", config: { name: "a" } }); - const resourceB = new BResource({ - id: "b", - config: { name: "b" }, - dependencies: { a: resourceA }, - }); - const resourceC = new CResource({ id: "c", config: { name: "c" } }); - - const reconciler = new Reconciler({ state, driftDetection: false }); - await reconciler.deploy([resourceA, resourceB, resourceC]); - - expect(Math.abs(marks.aStart - marks.cStart)).toBeLessThan(40); - expect(marks.bStart).toBeGreaterThanOrEqual(marks.aEnd); - }); - - it("detects drift using live read output and converges with update", async () => { - const updateSpy = vi.fn(async () => undefined); - const events: Array> = []; - const TestResource = createTestResourceClass({ - type: "test/service/drift", - read: async () => found({ name: "drifted" }), - update: updateSpy, - }); - - const state = createMemoryState({ - resource: { - rev: 1, - id: "resource", - groupId: -1, - groupType: "", - type: TestResource.type, - config: { name: "desired" }, - params: { name: "desired" }, - output: { name: "desired" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const reconciler = new Reconciler({ - state, - driftDetection: true, - emit: async (event) => { - events.push(event as unknown as Record); - }, - }); - await reconciler.deploy([ - new TestResource({ id: "resource", config: { name: "desired" } }), - ]); - - expect(updateSpy).toHaveBeenCalledOnce(); - expect(updateSpy.mock.calls[0]?.[1]).toEqual({ name: "desired" }); - expect(events).toContainEqual({ - level: "info", - event: "reconciler.drift.detected", - resourceId: "resource", - resourceType: TestResource.type, - diff: { name: "desired" }, - }); - }); - - it("deletes orphaned state entries by reconstructing from registry", async () => { - const deleteSpy = vi.fn(async () => undefined); - const OrphanResource = createTestResourceClass({ - type: "test/service/orphan", - delete: deleteSpy, - }); - - const state = createMemoryState({ - orphan: { - rev: 1, - id: "orphan", - groupId: -1, - groupType: "", - type: OrphanResource.type, - config: { name: "from-state" }, - params: { name: "from-state" }, - output: { name: "from-state" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const reconciler = new Reconciler({ - state, - registry: createResourceRegistry([OrphanResource]), - driftDetection: false, - }); - - await reconciler.deploy([]); - - expect(deleteSpy).toHaveBeenCalledOnce(); - expect(state.delete).toHaveBeenCalledWith("orphan", 1); - }); - - it("dryRun emits operation intent without applying side effects", async () => { - const createSpy = vi.fn(async () => ({ name: "new" })); - const deleteSpy = vi.fn(async () => undefined); - - const CreateResource = createTestResourceClass({ - type: "test/service/dry-run-create", - create: createSpy, - read: async () => found({ name: "new" }), - }); - const OrphanResource = createTestResourceClass({ - type: "test/service/dry-run-orphan", - delete: deleteSpy, - }); - - const state = createMemoryState({ - orphan: { - rev: 1, - id: "orphan", - groupId: -1, - groupType: "", - type: OrphanResource.type, - config: { name: "orphan" }, - params: { name: "orphan" }, - output: { name: "orphan" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const operationEvents: string[] = []; - const reconciler = new Reconciler({ - state, - registry: createResourceRegistry([OrphanResource]), - dryRun: true, - driftDetection: false, - emit: async (event) => { - if ("operation" in event) { - operationEvents.push( - `${event.operation}:${event.status}:${event.resourceId}`, - ); - } - }, - }); - - await reconciler.deploy([ - new CreateResource({ id: "new", config: { name: "new" } }), - ]); - - expect(createSpy).not.toHaveBeenCalled(); - expect(deleteSpy).not.toHaveBeenCalled(); - expect(state.update).not.toHaveBeenCalled(); - expect(state.delete).not.toHaveBeenCalled(); - expect(operationEvents).toContain("create:dry-run:new"); - expect(operationEvents).toContain("delete:dry-run:orphan"); - }); -}); - -describe("reconciler destroy + refresh", () => { - it("reads remote state after a delete conflict instead of deleting twice", async () => { - let remoteExists = true; - const deleteSpy = vi.fn(async () => { - remoteExists = false; - }); - const readSpy = vi.fn(async () => { - if (!remoteExists) { - throw new ResourceNotFoundError("resource is absent"); - } - return found({ name: "doomed" }); - }); - const DestroyResource = createTestResourceClass({ - type: "test/service/destroy-retry", - read: readSpy, - delete: deleteSpy, - }); - const state = createMemoryState({ - doomed: { - rev: 1, - id: "doomed", - groupId: -1, - groupType: "", - type: DestroyResource.type, - config: { name: "doomed" }, - params: { name: "doomed" }, - output: { name: "doomed" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - const deleteState = state.delete.getMockImplementation()!; - state.delete - .mockImplementationOnce(async () => { - state.store.doomed = { ...state.store.doomed!, rev: 2 }; - throw new RevConflict("doomed", 1, 2); - }) - .mockImplementation(deleteState); - - const reconciler = new Reconciler({ state }); - await reconciler.destroy([ - new DestroyResource({ id: "doomed", config: { name: "doomed" } }), - ]); - - expect(deleteSpy).toHaveBeenCalledOnce(); - expect(readSpy).toHaveBeenCalledOnce(); - expect(state.delete).toHaveBeenCalledTimes(2); - expect(state.store.doomed).toBeUndefined(); - }); - - it("holds a backend lease for the orphan snapshot", async () => { - const state = createMemoryState(); - const release = vi.fn(async () => undefined); - const lease = vi.fn(async () => ({ - scope: "reconciler:orphan-deletion", - expiresAt: new Date(Date.now() + 10_000).toISOString(), - renew: vi.fn(async () => new Date(Date.now() + 10_000).toISOString()), - release, - })); - const reconciler = new Reconciler({ - state: { ...state, lease }, - mutationLeaseTtl: 10_000, - }); - - await reconciler.refresh([]); - - expect(lease).toHaveBeenCalledWith("reconciler:orphan-deletion", 10_000); - expect(state.values).toHaveBeenCalledOnce(); - expect(release).toHaveBeenCalledOnce(); - }); - - it("destroys resources in reverse dependency order", async () => { - const destroyOrder: string[] = []; - const deleteA = vi.fn(async () => { - destroyOrder.push("a"); - }); - const deleteB = vi.fn(async () => { - destroyOrder.push("b"); - }); - const deleteC = vi.fn(async () => { - destroyOrder.push("c"); - }); - - const AResource = createTestResourceClass({ - type: "test/service/destroy-a", - delete: deleteA, - }); - const BResource = createTestResourceClass({ - type: "test/service/destroy-b", - delete: deleteB, - }); - const CResource = createTestResourceClass({ - type: "test/service/destroy-c", - delete: deleteC, - }); - - const resourceA = new AResource({ id: "a", config: { name: "a" } }); - const resourceB = new BResource({ - id: "b", - config: { name: "b" }, - dependencies: { a: resourceA }, - }); - const resourceC = new CResource({ - id: "c", - config: { name: "c" }, - dependencies: { b: resourceB }, - }); - - const state = createMemoryState({ - a: { - rev: 1, - id: "a", - groupId: -1, - groupType: "", - type: AResource.type, - config: { name: "a" }, - params: { name: "a" }, - output: { name: "a" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - b: { - rev: 1, - id: "b", - groupId: -1, - groupType: "", - type: BResource.type, - config: { name: "b" }, - params: { name: "b" }, - output: { name: "b" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - c: { - rev: 1, - id: "c", - groupId: -1, - groupType: "", - type: CResource.type, - config: { name: "c" }, - params: { name: "c" }, - output: { name: "c" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const reconciler = new Reconciler({ state }); - await reconciler.destroy([resourceA, resourceB, resourceC]); - - expect(destroyOrder).toEqual(["c", "b", "a"]); - expect(state.delete).toHaveBeenCalledWith("a", 1); - expect(state.delete).toHaveBeenCalledWith("b", 1); - expect(state.delete).toHaveBeenCalledWith("c", 1); - }); - - it("refresh removes orphan state entries", async () => { - const deleteSpy = vi.fn(async () => undefined); - const OrphanResource = createTestResourceClass({ - type: "test/service/refresh-orphan", - delete: deleteSpy, - }); - const KeepResource = createTestResourceClass({ - type: "test/service/refresh-keep", - }); - - const keep = new KeepResource({ id: "keep", config: { name: "keep" } }); - const state = createMemoryState({ - keep: { - rev: 1, - id: "keep", - groupId: -1, - groupType: "", - type: KeepResource.type, - config: { name: "keep" }, - params: { name: "keep" }, - output: { name: "keep" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - orphan: { - rev: 1, - id: "orphan", - groupId: -1, - groupType: "", - type: OrphanResource.type, - config: { name: "orphan" }, - params: { name: "orphan" }, - output: { name: "orphan" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const reconciler = new Reconciler({ - state, - registry: createResourceRegistry([OrphanResource]), - }); - - await reconciler.refresh([keep]); - - expect(deleteSpy).toHaveBeenCalledOnce(); - expect(state.delete).toHaveBeenCalledWith("orphan", 1); - expect(state.delete).not.toHaveBeenCalledWith("keep", expect.anything()); - }); - - it("destroy and refresh dryRun emit operation events without side effects", async () => { - const deleteSpy = vi.fn(async () => undefined); - const DestroyResource = createTestResourceClass({ - type: "test/service/dry-run-destroy", - delete: deleteSpy, - }); - const OrphanResource = createTestResourceClass({ - type: "test/service/dry-run-refresh", - delete: deleteSpy, - }); - - const destroyResource = new DestroyResource({ - id: "destroy-me", - config: { name: "destroy-me" }, - }); - - const state = createMemoryState({ - "destroy-me": { - rev: 1, - id: "destroy-me", - groupId: -1, - groupType: "", - type: DestroyResource.type, - config: { name: "destroy-me" }, - params: { name: "destroy-me" }, - output: { name: "destroy-me" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - orphan: { - rev: 1, - id: "orphan", - groupId: -1, - groupType: "", - type: OrphanResource.type, - config: { name: "orphan" }, - params: { name: "orphan" }, - output: { name: "orphan" }, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }, - }); - - const operationEvents: string[] = []; - const reconciler = new Reconciler({ - state, - dryRun: true, - registry: createResourceRegistry([OrphanResource]), - emit: async (event) => { - if ("operation" in event) { - operationEvents.push( - `${event.operation}:${event.status}:${event.resourceId}`, - ); - } - }, - }); - - await reconciler.destroy([destroyResource]); - await reconciler.refresh([destroyResource]); - - expect(deleteSpy).not.toHaveBeenCalled(); - expect(state.delete).not.toHaveBeenCalled(); - expect(operationEvents).toContain("delete:dry-run:destroy-me"); - expect(operationEvents).toContain("delete:dry-run:orphan"); - }); -}); diff --git a/packages/reconciler/test/reconciler.plan.test.ts b/packages/reconciler/test/reconciler.plan.test.ts deleted file mode 100644 index d98e55a..0000000 --- a/packages/reconciler/test/reconciler.plan.test.ts +++ /dev/null @@ -1,463 +0,0 @@ -import { describe, expect, it, vi } from "vitest"; -import { - ResourceNotFoundError, - ResourceOperationPendingError, - resource, - type BaseResource, -} from "@notation/resource"; -import type { StateNode } from "@notation/state"; -import { Reconciler, UNKNOWN_AFTER_APPLY } from "../src"; - -function createMemoryState(initial: Record = {}) { - const store: Record = { ...initial }; - - return { - store, - get: vi.fn(async (id: string) => store[id]), - update: vi.fn( - async (id: string, expectedRev: number, patch: Partial) => { - store[id] = { - ...(store[id] ?? {}), - ...patch, - } as StateNode; - }, - ), - delete: vi.fn(async (id: string) => { - delete store[id]; - }), - values: vi.fn(async () => Object.values(store)), - lease: vi.fn(async (scope: string, ttl: number) => ({ - scope, - expiresAt: new Date(Date.now() + ttl).toISOString(), - renew: vi.fn(async (nextTtl: number) => - new Date(Date.now() + nextTtl).toISOString(), - ), - release: vi.fn(async () => undefined), - })), - }; -} - -function createTestResourceClass(opts: { - type: `${string}/${string}/${string}`; - create?: ( - params: Record, - ) => Promise | void>; - read?: (key: Record) => Promise>; - update?: ( - key: Record, - patch: Record, - params: Record, - state: Record, - ) => Promise; - delete?: ( - key: Record, - state: Record, - ) => Promise; -}) { - return resource({ type: opts.type }) - .defineSchema({ - name: { - presence: "required", - propertyType: "param", - valueType: "string" as any, - }, - tag: { - presence: "optional", - propertyType: "param", - valueType: "string" as any, - }, - }) - .defineOperations({ - create: opts.create ?? (async () => ({})), - read: opts.read, - update: opts.update, - delete: opts.delete ?? (async () => undefined), - }); -} - -function createStateNode( - id: string, - type: string, - params: Record, - output: Record = params, -): StateNode { - return { - id, - groupId: -1, - groupType: "", - type, - config: params, - params, - output, - lastOperation: "create", - lastOperationAt: new Date().toISOString(), - }; -} - -describe("reconciler plan", () => { - it("plans create for resources without state", async () => { - const TestResource = createTestResourceClass({ - type: "test/service/plan-create", - }); - - const state = createMemoryState(); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([ - new TestResource({ id: "new", config: { name: "new" } }), - ]); - - expect(plan.nodes).toEqual([ - { - id: "new", - type: TestResource.type, - decision: "create", - params: { name: "new" }, - dependsOn: [], - }, - ]); - }); - - it("plans update with the detailed diff that justified it", async () => { - const TestResource = createTestResourceClass({ - type: "test/service/plan-update", - }); - - const state = createMemoryState({ - existing: createStateNode("existing", "test/service/plan-update", { - name: "old", - tag: "keep", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([ - new TestResource({ id: "existing", config: { name: "new" } }), - ]); - - expect(plan.nodes).toEqual([ - { - id: "existing", - type: TestResource.type, - decision: "update", - diff: { - added: {}, - deleted: { tag: null }, - updated: { name: "new" }, - }, - params: { name: "new" }, - dependsOn: [], - }, - ]); - }); - - it("plans noop when params match state", async () => { - const TestResource = createTestResourceClass({ - type: "test/service/plan-noop", - }); - - const state = createMemoryState({ - unchanged: createStateNode("unchanged", "test/service/plan-noop", { - name: "same", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([ - new TestResource({ id: "unchanged", config: { name: "same" } }), - ]); - - expect(plan.nodes[0]).toMatchObject({ id: "unchanged", decision: "noop" }); - }); - - it("plans drift-update from live read output when drift detection is on", async () => { - const readSpy = vi.fn(async () => ({ name: "drifted" })); - const TestResource = createTestResourceClass({ - type: "test/service/plan-drift-update", - read: readSpy, - }); - - const state = createMemoryState({ - resource: createStateNode("resource", "test/service/plan-drift-update", { - name: "desired", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: true }); - - const plan = await reconciler.plan([ - new TestResource({ id: "resource", config: { name: "desired" } }), - ]); - - expect(readSpy).toHaveBeenCalledOnce(); - expect(plan.nodes[0]).toEqual({ - id: "resource", - type: TestResource.type, - decision: "drift-update", - diff: { - added: {}, - deleted: {}, - updated: { name: "desired" }, - }, - params: { name: "desired" }, - dependsOn: [], - }); - }); - - it("plans drift-recreate when the remote resource is gone", async () => { - const TestResource = createTestResourceClass({ - type: "test/service/plan-drift-recreate", - read: async () => { - throw new ResourceNotFoundError("resource is absent"); - }, - }); - - const state = createMemoryState({ - resource: createStateNode( - "resource", - "test/service/plan-drift-recreate", - { name: "desired" }, - ), - }); - const reconciler = new Reconciler({ state, driftDetection: true }); - - const plan = await reconciler.plan([ - new TestResource({ id: "resource", config: { name: "desired" } }), - ]); - - expect(plan.nodes[0]).toMatchObject({ - id: "resource", - decision: "drift-recreate", - }); - }); - - it("waits for a pending read before planning", async () => { - let attempts = 0; - const TestResource = createTestResourceClass({ - type: "test/service/plan-not-ready", - read: async () => { - attempts += 1; - if (attempts === 1) { - throw new ResourceOperationPendingError( - "Waiting for Lambda to become active", - { retryAfterMs: 0 }, - ); - } - return { name: "desired" }; - }, - }); - - const state = createMemoryState({ - resource: createStateNode("resource", "test/service/plan-not-ready", { - name: "desired", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: true }); - - const plan = await reconciler.plan([ - new TestResource({ id: "resource", config: { name: "desired" } }), - ]); - - expect(plan.nodes[0]).toMatchObject({ - id: "resource", - decision: "noop", - }); - expect(attempts).toBe(2); - }); - - it("skips remote reads when drift detection is off", async () => { - const readSpy = vi.fn(async () => ({ name: "drifted" })); - const TestResource = createTestResourceClass({ - type: "test/service/plan-no-read", - read: readSpy, - }); - - const state = createMemoryState({ - resource: createStateNode("resource", "test/service/plan-no-read", { - name: "desired", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: false }); - - await reconciler.plan([ - new TestResource({ id: "resource", config: { name: "desired" } }), - ]); - - expect(readSpy).not.toHaveBeenCalled(); - }); - - it("plans delete-orphan for state nodes without a matching resource", async () => { - const state = createMemoryState({ - orphan: createStateNode("orphan", "test/service/plan-orphan", { - name: "orphan", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([]); - - expect(plan.nodes).toEqual([ - { - id: "orphan", - type: "test/service/plan-orphan", - decision: "delete-orphan", - params: { name: "orphan" }, - dependsOn: [], - }, - ]); - }); - - it("populates dependsOn from resource dependencies", async () => { - const AResource = createTestResourceClass({ - type: "test/service/plan-dep-a", - }); - const BResource = createTestResourceClass({ - type: "test/service/plan-dep-b", - }); - - const resourceA = new AResource({ id: "a", config: { name: "a" } }); - const resourceB = new BResource({ - id: "b", - config: { name: "b" }, - dependencies: { a: resourceA }, - }); - - const state = createMemoryState(); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([resourceA, resourceB]); - - const nodeB = plan.nodes.find((node) => node.id === "b"); - expect(nodeB?.dependsOn).toEqual(["a"]); - }); - - it("marks params derived from uncreated dependencies as unknown after apply", async () => { - const AResource = createTestResourceClass({ - type: "test/service/plan-unknown-a", - }); - const BResource = createTestResourceClass({ - type: "test/service/plan-unknown-b", - }) - .requireDependencies<{ a: BaseResource }>() - .deriveParams(({ deps }) => ({ - name: (deps.a.output as { name: string }).name, - })); - - const resourceA = new AResource({ id: "a", config: { name: "a" } }); - const resourceB = new BResource({ - id: "b", - config: { tag: "known" }, - dependencies: { a: resourceA }, - }); - - const state = createMemoryState(); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([resourceA, resourceB]); - - const nodeB = plan.nodes.find((node) => node.id === "b"); - expect(nodeB).toMatchObject({ - decision: "create", - params: { - name: UNKNOWN_AFTER_APPLY, - tag: "known", - }, - }); - }); - - it("does not disguise parameter derivation failures as unknown values", async () => { - const TestResource = resource({ - type: "test/service/plan-derive-failure", - }) - .defineSchema({ - name: { - presence: "required", - propertyType: "param", - valueType: "string" as any, - }, - }) - .defineOperations({ - create: async () => ({}), - delete: async () => undefined, - deriveParams: () => { - throw new Error("invalid derived configuration"); - }, - }); - - const reconciler = new Reconciler({ - state: createMemoryState(), - driftDetection: false, - }); - - await expect( - reconciler.plan([new TestResource({ id: "broken" })]), - ).rejects.toThrow("invalid derived configuration"); - }); - - it("produces a JSON-round-trippable plan", async () => { - const CreateResource = createTestResourceClass({ - type: "test/service/plan-json-create", - }); - const UpdateResource = createTestResourceClass({ - type: "test/service/plan-json-update", - }); - - const state = createMemoryState({ - existing: createStateNode("existing", "test/service/plan-json-update", { - name: "old", - tag: "gone", - }), - orphan: createStateNode("orphan", "test/service/plan-json-orphan", { - name: "orphan", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: false }); - - const plan = await reconciler.plan([ - new CreateResource({ id: "new", config: { name: "new" } }), - new UpdateResource({ id: "existing", config: { name: "new" } }), - ]); - - expect(JSON.parse(JSON.stringify(plan))).toStrictEqual(plan); - }); - - it("performs no state writes or resource operations", async () => { - const createSpy = vi.fn(async () => ({ name: "new" })); - const updateSpy = vi.fn(async () => undefined); - const deleteSpy = vi.fn(async () => undefined); - - const CreateResource = createTestResourceClass({ - type: "test/service/plan-pure-create", - create: createSpy, - update: updateSpy, - delete: deleteSpy, - }); - const UpdateResource = createTestResourceClass({ - type: "test/service/plan-pure-update", - create: createSpy, - update: updateSpy, - delete: deleteSpy, - read: async () => ({ name: "drifted" }), - }); - - const state = createMemoryState({ - existing: createStateNode("existing", "test/service/plan-pure-update", { - name: "same", - }), - orphan: createStateNode("orphan", "test/service/plan-pure-orphan", { - name: "orphan", - }), - }); - const reconciler = new Reconciler({ state, driftDetection: true }); - - await reconciler.plan([ - new CreateResource({ id: "new", config: { name: "new" } }), - new UpdateResource({ id: "existing", config: { name: "same" } }), - ]); - - expect(createSpy).not.toHaveBeenCalled(); - expect(updateSpy).not.toHaveBeenCalled(); - expect(deleteSpy).not.toHaveBeenCalled(); - expect(state.update).not.toHaveBeenCalled(); - expect(state.delete).not.toHaveBeenCalled(); - }); -}); diff --git a/packages/reconciler/test/resource-registry.test.ts b/packages/reconciler/test/resource-registry.test.ts index dd63f48..de3fdf8 100644 --- a/packages/reconciler/test/resource-registry.test.ts +++ b/packages/reconciler/test/resource-registry.test.ts @@ -9,14 +9,14 @@ import { const TestResourceA = resource({ type: "test/service/a" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, delete: async () => undefined, }); const TestResourceB = resource({ type: "test/service/b" }) .defineSchema({}) .defineOperations({ - create: async () => ({}), + create: (async () => ({})) as any, delete: async () => undefined, }); diff --git a/packages/reconciler/tsup.config.ts b/packages/reconciler/tsup.config.ts index f0ac238..2834ee4 100644 --- a/packages/reconciler/tsup.config.ts +++ b/packages/reconciler/tsup.config.ts @@ -1,7 +1,7 @@ import { defineConfig } from "tsup"; export default defineConfig({ - entry: ["src/index.ts"], + entry: ["src/index.ts", "src/durable/index.ts"], dts: true, format: ["esm"], }); diff --git a/packages/state-sqlite/package.json b/packages/state-sqlite/package.json deleted file mode 100644 index 1ad01b7..0000000 --- a/packages/state-sqlite/package.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "type": "module", - "name": "@notation/state-sqlite", - "version": "0.1.0", - "main": "./dist/index.js", - "types": "./dist/index.d.ts", - "files": [ - "dist" - ], - "scripts": { - "build": "tsup --clean", - "dev": "tsup --watch" - }, - "dependencies": { - "@notation/state": "workspace:*" - }, - "devDependencies": { - "@types/node": "^22.13.4" - } -} diff --git a/packages/state-sqlite/src/index.ts b/packages/state-sqlite/src/index.ts deleted file mode 100644 index 057628f..0000000 --- a/packages/state-sqlite/src/index.ts +++ /dev/null @@ -1,195 +0,0 @@ -import { randomUUID } from "node:crypto"; -import { mkdirSync } from "node:fs"; -import { dirname } from "node:path"; -import { DatabaseSync } from "node:sqlite"; -import { - LeaseConflict, - RevConflict, - type Lease, - type StateBackend, - type StateNode, -} from "@notation/state"; - -export class SqliteStateBackend implements StateBackend { - readonly #database: DatabaseSync; - - constructor(path: string) { - mkdirSync(dirname(path), { recursive: true }); - this.#database = new DatabaseSync(path); - this.#database.exec("PRAGMA busy_timeout = 5000"); - this.#database.exec(` - CREATE TABLE IF NOT EXISTS resources ( - id TEXT PRIMARY KEY, - rev INTEGER NOT NULL, - value TEXT NOT NULL - ) - `); - this.#database.exec(` - CREATE TABLE IF NOT EXISTS resource_leases ( - scope TEXT PRIMARY KEY, - owner TEXT NOT NULL, - expires_at INTEGER NOT NULL - ) - `); - } - - close(): void { - this.#database.close(); - } - - async get(id: string): Promise { - const row = this.#database - .prepare("SELECT value FROM resources WHERE id = ?") - .get(id) as { value: string } | undefined; - return row ? (JSON.parse(row.value) as StateNode) : undefined; - } - - async has(id: string): Promise { - return Boolean( - this.#database - .prepare("SELECT 1 FROM resources WHERE id = ?") - .get(id), - ); - } - - async update( - id: string, - expectedRev: number, - patch: Partial, - ): Promise<{ rev: number }> { - this.#database.exec("BEGIN IMMEDIATE"); - try { - const current = await this.get(id); - // A missing record counts as rev 0, so expectedRev: 0 = "must not exist". - if ((current?.rev ?? 0) !== expectedRev) { - throw new RevConflict(id, expectedRev, current?.rev); - } - - const rev = (current?.rev ?? 0) + 1; - const node = { ...current, ...patch, rev } as StateNode; - if (current) { - const result = this.#database - .prepare( - "UPDATE resources SET rev = ?, value = ? WHERE id = ? AND rev = ?", - ) - .run(rev, JSON.stringify(node), id, current.rev); - if (result.changes !== 1) { - const actual = await this.get(id); - throw new RevConflict(id, current.rev, actual?.rev); - } - } else { - this.#database - .prepare( - "INSERT INTO resources (id, rev, value) VALUES (?, ?, ?)", - ) - .run(id, rev, JSON.stringify(node)); - } - this.#database.exec("COMMIT"); - return { rev }; - } catch (error) { - this.#database.exec("ROLLBACK"); - throw error; - } - } - - async delete(id: string, expectedRev: number): Promise { - const current = await this.get(id); - if ((current?.rev ?? 0) !== expectedRev) { - throw new RevConflict(id, expectedRev, current?.rev); - } - if (!current) return; - - const result = this.#database - .prepare("DELETE FROM resources WHERE id = ? AND rev = ?") - .run(id, current.rev); - if (result.changes !== 1) { - const actual = await this.get(id); - throw new RevConflict(id, current.rev, actual?.rev); - } - } - - async values(): Promise { - const rows = this.#database - .prepare("SELECT value FROM resources ORDER BY id") - .all() as { value: string }[]; - return rows.map(({ value }) => JSON.parse(value) as StateNode); - } - - async lease(scope: string, ttl: number): Promise { - if (!Number.isFinite(ttl) || ttl <= 0) { - throw new RangeError( - "Lease TTL must be a positive number of milliseconds", - ); - } - - const owner = randomUUID(); - const expiresAtMs = Date.now() + ttl; - this.#database.exec("BEGIN IMMEDIATE"); - try { - this.#database - .prepare( - "DELETE FROM resource_leases WHERE scope = ? AND expires_at <= ?", - ) - .run(scope, Date.now()); - const current = this.#database - .prepare("SELECT expires_at FROM resource_leases WHERE scope = ?") - .get(scope) as { expires_at: number } | undefined; - if (current) { - throw new LeaseConflict( - scope, - new Date(current.expires_at).toISOString(), - ); - } - this.#database - .prepare( - "INSERT INTO resource_leases (scope, owner, expires_at) VALUES (?, ?, ?)", - ) - .run(scope, owner, expiresAtMs); - this.#database.exec("COMMIT"); - } catch (error) { - this.#database.exec("ROLLBACK"); - throw error; - } - - let released = false; - let currentExpiresAtMs = expiresAtMs; - return { - scope, - get expiresAt() { - return new Date(currentExpiresAtMs).toISOString(); - }, - renew: async (nextTtl) => { - if (!Number.isFinite(nextTtl) || nextTtl <= 0) { - throw new RangeError( - "Lease TTL must be a positive number of milliseconds", - ); - } - const now = Date.now(); - const nextExpiresAtMs = now + nextTtl; - const result = this.#database - .prepare( - "UPDATE resource_leases SET expires_at = ? WHERE scope = ? AND owner = ? AND expires_at > ?", - ) - .run(nextExpiresAtMs, scope, owner, now); - if (result.changes !== 1) { - const current = this.#database - .prepare("SELECT expires_at FROM resource_leases WHERE scope = ?") - .get(scope) as { expires_at: number } | undefined; - throw new LeaseConflict( - scope, - new Date(current?.expires_at ?? 0).toISOString(), - ); - } - currentExpiresAtMs = nextExpiresAtMs; - return new Date(nextExpiresAtMs).toISOString(); - }, - release: async () => { - if (released) return; - this.#database - .prepare("DELETE FROM resource_leases WHERE scope = ? AND owner = ?") - .run(scope, owner); - released = true; - }, - }; - } -} diff --git a/packages/state-sqlite/src/node-sqlite.d.ts b/packages/state-sqlite/src/node-sqlite.d.ts deleted file mode 100644 index 04691f7..0000000 --- a/packages/state-sqlite/src/node-sqlite.d.ts +++ /dev/null @@ -1,20 +0,0 @@ -declare module "node:sqlite" { - export type StatementResult = { changes: number | bigint }; - export class StatementSync { - get(...values: unknown[]): unknown; - all(...values: unknown[]): unknown[]; - run(...values: unknown[]): StatementResult; - } - export class DatabaseSync { - constructor(path: string); - exec(sql: string): void; - prepare(sql: string): StatementSync; - close(): void; - } -} - -// The shared tsconfig does not load @types/node, so declare the one export -// this package uses. -declare module "node:crypto" { - export function randomUUID(): string; -} diff --git a/packages/state-sqlite/test/state-sqlite.test.ts b/packages/state-sqlite/test/state-sqlite.test.ts deleted file mode 100644 index 31eb35d..0000000 --- a/packages/state-sqlite/test/state-sqlite.test.ts +++ /dev/null @@ -1,122 +0,0 @@ -import { mkdtemp, rm } from "node:fs/promises"; -import { spawn } from "node:child_process"; -import { once } from "node:events"; -import { tmpdir } from "node:os"; -import path from "node:path"; -import { afterEach, describe, expect, it } from "vitest"; -import { SqliteStateBackend } from "../src"; - -const cleanups: (() => Promise)[] = []; -afterEach(async () => - Promise.all(cleanups.splice(0).map((cleanup) => cleanup())), -); - -async function createBackend() { - const directory = await mkdtemp(path.join(tmpdir(), "notation-sqlite-")); - const backend = new SqliteStateBackend(path.join(directory, "state.db")); - cleanups.push(async () => { - backend.close(); - await rm(directory, { recursive: true, force: true }); - }); - return backend; -} - -describe("SqliteStateBackend", () => { - it("persists revisions and enforces compare-and-swap", async () => { - const backend = await createBackend(); - await expect( - backend.update("service", 0, { - id: "service", - type: "test/service/main", - config: {}, - params: {}, - output: {}, - lastOperation: "create", - lastOperationAt: "2026-07-15T00:00:00.000Z", - }), - ).resolves.toEqual({ rev: 1 }); - await expect( - backend.update("service", 1, { output: { ready: true } }), - ).resolves.toEqual({ - rev: 2, - }); - await expect(backend.delete("service", 1)).rejects.toMatchObject({ - name: "RevConflict", - actualRev: 2, - }); - }); - - it("coordinates leases across backend instances and releases by owner", async () => { - const directory = await mkdtemp( - path.join(tmpdir(), "notation-sqlite-lease-"), - ); - const databasePath = path.join(directory, "state.db"); - const first = new SqliteStateBackend(databasePath); - const second = new SqliteStateBackend(databasePath); - cleanups.push(async () => { - first.close(); - second.close(); - await rm(directory, { recursive: true, force: true }); - }); - - const lease = await first.lease("orphans", 10_000); - await expect(second.lease("orphans", 10_000)).rejects.toMatchObject({ - name: "LeaseConflict", - scope: "orphans", - }); - const firstExpiry = lease.expiresAt; - await lease.renew(20_000); - expect(lease.expiresAt).not.toBe(firstExpiry); - await lease.release(); - const nextLease = await second.lease("orphans", 10_000); - expect(nextLease).toMatchObject({ scope: "orphans" }); - await nextLease.release(); - }); - - it("waits for a concurrent writer instead of raising database locked", async () => { - const directory = await mkdtemp( - path.join(tmpdir(), "notation-sqlite-busy-"), - ); - const databasePath = path.join(directory, "state.db"); - const backend = new SqliteStateBackend(databasePath); - cleanups.push(async () => { - backend.close(); - await rm(directory, { recursive: true, force: true }); - }); - - const blocker = spawn( - process.execPath, - [ - "--input-type=module", - "--eval", - ` - import { DatabaseSync } from "node:sqlite"; - const database = new DatabaseSync(process.argv[1]); - database.exec("BEGIN IMMEDIATE"); - process.stdout.write("locked\\n"); - setTimeout(() => { - database.exec("ROLLBACK"); - database.close(); - }, 100); - `, - databasePath, - ], - { stdio: ["ignore", "pipe", "inherit"] }, - ); - const blockerExited = once(blocker, "exit"); - await once(blocker.stdout!, "data"); - - await expect( - backend.update("service", 0, { - id: "service", - type: "test/service/main", - config: {}, - params: {}, - output: {}, - lastOperation: "create", - lastOperationAt: "2026-07-15T00:00:00.000Z", - }), - ).resolves.toEqual({ rev: 1 }); - await blockerExited; - }); -}); diff --git a/packages/state-sqlite/tsconfig.json b/packages/state-sqlite/tsconfig.json deleted file mode 100644 index 13487e3..0000000 --- a/packages/state-sqlite/tsconfig.json +++ /dev/null @@ -1,7 +0,0 @@ -{ - "extends": "tsconfig/base.json", - "compilerOptions": { - "types": ["node"] - }, - "include": ["src", "test"] -} diff --git a/packages/state-sqlite/tsup.config.ts b/packages/state-sqlite/tsup.config.ts deleted file mode 100644 index a61d2e2..0000000 --- a/packages/state-sqlite/tsup.config.ts +++ /dev/null @@ -1,10 +0,0 @@ -import { defineConfig } from "tsup"; - -export default defineConfig({ - entry: ["src/index.ts"], - format: ["esm"], - dts: true, - sourcemap: true, - // node:sqlite only resolves with the node: prefix; don't let tsup strip it. - removeNodeProtocol: false, -}); diff --git a/packages/state/package.json b/packages/state/package.json index 8a49f6f..2c81b57 100644 --- a/packages/state/package.json +++ b/packages/state/package.json @@ -11,9 +11,6 @@ "build": "tsup --clean", "dev": "tsup --watch" }, - "dependencies": { - "@notation/utils": "workspace:*" - }, "devDependencies": { "@types/node": "^22.13.4" } diff --git a/packages/state/src/conflicts.ts b/packages/state/src/conflicts.ts index 81559dc..23494ad 100644 --- a/packages/state/src/conflicts.ts +++ b/packages/state/src/conflicts.ts @@ -1,24 +1,18 @@ -export class RevConflict extends Error { - readonly name = "RevConflict"; +/** + * A conditional state write or removal found the record moved past the + * version it was read at. Nothing catches this: it fails the workflow, and + * the constructor arguments exist to name the losing write in the message. + */ +export class VersionConflict extends Error { + readonly name = "VersionConflict"; constructor( - readonly id: string, - readonly expectedRev: number, - readonly actualRev: number | undefined, + id: string, + expectedVersion: number, + actualVersion: number | undefined, ) { super( - `State revision conflict for ${id}: expected ${expectedRev}, got ${actualRev ?? "missing"}`, + `State version conflict for ${id}: expected ${expectedVersion}, got ${actualVersion ?? "missing"}`, ); } } - -export class LeaseConflict extends Error { - readonly name = "LeaseConflict"; - - constructor( - readonly scope: string, - readonly expiresAt: string, - ) { - super(`State lease conflict for ${scope}: held until ${expiresAt}`); - } -} diff --git a/packages/state/src/state.ts b/packages/state/src/state.ts index d0c10c1..dca02cf 100644 --- a/packages/state/src/state.ts +++ b/packages/state/src/state.ts @@ -1,58 +1,27 @@ -import { randomUUID } from "node:crypto"; -import { - mkdir, - readFile, - rename, - stat, - unlink, - writeFile, -} from "node:fs/promises"; -import path from "node:path"; -import { setTimeout as sleep } from "node:timers/promises"; -import { isErrorWithCode } from "@notation/utils"; -import { LeaseConflict, RevConflict } from "./conflicts"; - export type StateNode = { - rev: number; + /** The backing store's version of the record, counted from zero. */ + version: number; id: string; type: string; config: Record; params: Record; output: Record; - lastOperation: "drift" | "create" | "update" | "delete"; + lastOperation: "create" | "update"; lastOperationAt: string; [key: string]: unknown; }; +/** + * Read-only: state writes happen inside the durable workflow, through the + * store handle, so each write is stamped with the step that made it. + */ export interface StateBackend { get(id: string): Promise; - has(id: string): Promise; - /** - * The stored revision must match expectedRev. A missing record counts as - * revision 0, so expectedRev: 0 asserts that the record does not exist yet. - */ - update( - id: string, - expectedRev: number, - patch: Partial, - ): Promise<{ rev: number }>; - delete(id: string, expectedRev: number): Promise; values(): Promise; - lease(scope: string, ttl: number): Promise; -} - -export interface Lease { - readonly scope: string; - readonly expiresAt: string; - renew(ttl: number): Promise; - release(): Promise; } -export type State = StateBackend; - export class MemoryStateBackend implements StateBackend { #state: Record; - #leases = new Map(); constructor(initialState: Record = {}) { this.#state = cloneAsPersistedState(initialState); @@ -63,35 +32,6 @@ export class MemoryStateBackend implements StateBackend { return state[id]; } - async has(id: string): Promise { - const state = await this.readState(); - return id in state; - } - - async update( - id: string, - expectedRev: number, - patch: Partial, - ): Promise<{ rev: number }> { - const state = await this.readState(); - assertExpectedRev(id, state[id], expectedRev); - const rev = (state[id]?.rev ?? 0) + 1; - state[id] = { - ...state[id], - ...patch, - rev, - } as StateNode; - await this.writeState(state); - return { rev }; - } - - async delete(id: string, expectedRev: number): Promise { - const state = await this.readState(); - assertExpectedRev(id, state[id], expectedRev); - delete state[id]; - await this.writeState(state); - } - async values(): Promise { const state = await this.readState(); return Object.entries(state) @@ -109,271 +49,13 @@ export class MemoryStateBackend implements StateBackend { .map(([, value]) => value); } - async lease(scope: string, ttl: number): Promise { - assertLeaseTtl(ttl); - const now = Date.now(); - const current = this.#leases.get(scope); - if (current && current.expiresAtMs > now) { - throw new LeaseConflict( - scope, - new Date(current.expiresAtMs).toISOString(), - ); - } - - const owner = randomUUID(); - let expiresAtMs = now + ttl; - this.#leases.set(scope, { owner, expiresAtMs }); - - return { - scope, - get expiresAt() { - return new Date(expiresAtMs).toISOString(); - }, - renew: async (nextTtl) => { - assertLeaseTtl(nextTtl); - const held = this.#leases.get(scope); - if (!held || held.owner !== owner || held.expiresAtMs <= Date.now()) { - throw new LeaseConflict( - scope, - new Date(held?.expiresAtMs ?? 0).toISOString(), - ); - } - expiresAtMs = Date.now() + nextTtl; - held.expiresAtMs = expiresAtMs; - return new Date(expiresAtMs).toISOString(); - }, - release: async () => { - if (this.#leases.get(scope)?.owner === owner) - this.#leases.delete(scope); - }, - }; - } - private async readState(): Promise> { return cloneAsPersistedState(this.#state); } - - private async writeState(state: Record): Promise { - this.#state = cloneAsPersistedState(state); - } -} - -const FILE_LOCK_STALE_MS = 10_000; -const FILE_LOCK_TIMEOUT_MS = 5_000; -const FILE_LOCK_RETRY_MS = 25; - -export class FileStateBackend implements StateBackend { - constructor(private readonly stateFilePath: string) {} - - async get(id: string): Promise { - const state = await this.readState(); - return state[id]; - } - - async has(id: string): Promise { - const state = await this.readState(); - return id in state; - } - - async update( - id: string, - expectedRev: number, - patch: Partial, - ): Promise<{ rev: number }> { - return this.withLock(async () => { - const state = await this.readState(); - assertExpectedRev(id, state[id], expectedRev); - const rev = (state[id]?.rev ?? 0) + 1; - state[id] = { - ...state[id], - ...patch, - rev, - } as StateNode; - await this.writeState(state); - return { rev }; - }); - } - - async delete(id: string, expectedRev: number): Promise { - await this.withLock(async () => { - const state = await this.readState(); - assertExpectedRev(id, state[id], expectedRev); - delete state[id]; - await this.writeState(state); - }); - } - - async values(): Promise { - const state = await this.readState(); - return Object.values(state); - } - - async lease(scope: string, ttl: number): Promise { - assertLeaseTtl(ttl); - const leaseFilePath = `${this.stateFilePath}.${encodeURIComponent(scope)}.lease`; - const owner = randomUUID(); - let expiresAtMs: number; - await mkdir(path.dirname(this.stateFilePath), { recursive: true }); - - for (;;) { - expiresAtMs = Date.now() + ttl; - try { - await writeFile(leaseFilePath, JSON.stringify({ owner, expiresAtMs }), { - flag: "wx", - }); - break; - } catch (error) { - if (!isErrorWithCode(error, "EEXIST")) throw error; - const current = await readFileLease(leaseFilePath); - if (!current || current.expiresAtMs <= Date.now()) { - await unlink(leaseFilePath).catch(() => undefined); - continue; - } - throw new LeaseConflict( - scope, - new Date(current.expiresAtMs).toISOString(), - ); - } - } - - return { - scope, - get expiresAt() { - return new Date(expiresAtMs).toISOString(); - }, - renew: async (nextTtl) => { - assertLeaseTtl(nextTtl); - const current = await readFileLease(leaseFilePath); - if ( - !current || - current.owner !== owner || - current.expiresAtMs <= Date.now() - ) { - throw new LeaseConflict( - scope, - new Date(current?.expiresAtMs ?? 0).toISOString(), - ); - } - expiresAtMs = Date.now() + nextTtl; - await writeFile(leaseFilePath, JSON.stringify({ owner, expiresAtMs })); - return new Date(expiresAtMs).toISOString(); - }, - release: async () => { - const current = await readFileLease(leaseFilePath); - if (current?.owner === owner) { - await unlink(leaseFilePath).catch(() => undefined); - } - }, - }; - } - - private async readState(): Promise> { - try { - const file = await readFile(this.stateFilePath, "utf8"); - return JSON.parse(file) as Record; - } catch (error) { - if (isErrorWithCode(error, "ENOENT")) { - return {}; - } - - throw error; - } - } - - /** - * The read-check-write in update/delete is only safe if no other process - * interleaves, so writers hold an exclusive lock file. A lock older than - * FILE_LOCK_STALE_MS is treated as abandoned by a crashed process. - */ - private async withLock(fn: () => Promise): Promise { - const lockFilePath = `${this.stateFilePath}.lock`; - await mkdir(path.dirname(this.stateFilePath), { recursive: true }); - - const deadline = Date.now() + FILE_LOCK_TIMEOUT_MS; - for (;;) { - try { - await writeFile( - lockFilePath, - JSON.stringify({ pid: process.pid, acquiredAt: Date.now() }), - { flag: "wx" }, - ); - break; - } catch (error) { - if (!isErrorWithCode(error, "EEXIST")) throw error; - const lockStat = await stat(lockFilePath).catch(() => undefined); - if (lockStat && Date.now() - lockStat.mtimeMs > FILE_LOCK_STALE_MS) { - await unlink(lockFilePath).catch(() => undefined); - continue; - } - if (Date.now() > deadline) { - throw new Error( - `Timed out acquiring state lock at ${lockFilePath}; delete it if no other deploy is running`, - ); - } - await sleep(FILE_LOCK_RETRY_MS); - } - } - - try { - return await fn(); - } finally { - await unlink(lockFilePath).catch(() => undefined); - } - } - - private async writeState(state: Record): Promise { - const directory = path.dirname(this.stateFilePath); - await mkdir(directory, { recursive: true }); - - const tempFilePath = path.join( - directory, - `${path.basename(this.stateFilePath)}.${randomUUID()}.tmp`, - ); - - const serialized = `${JSON.stringify(state, null, 2)}\n`; - - await writeFile(tempFilePath, serialized, "utf8"); - - try { - await rename(tempFilePath, this.stateFilePath); - } catch (error) { - await unlink(tempFilePath).catch(() => undefined); - throw error; - } - } -} - -// A missing record counts as rev 0, so expectedRev: 0 means "must not exist". -function assertExpectedRev( - id: string, - node: StateNode | undefined, - expectedRev: number, -): void { - if ((node?.rev ?? 0) !== expectedRev) { - throw new RevConflict(id, expectedRev, node?.rev); - } -} - -function assertLeaseTtl(ttl: number): void { - if (!Number.isFinite(ttl) || ttl <= 0) { - throw new RangeError("Lease TTL must be a positive number of milliseconds"); - } -} - -type FileLeaseRecord = { owner: string; expiresAtMs: number }; - -async function readFileLease( - filePath: string, -): Promise { - try { - return JSON.parse(await readFile(filePath, "utf8")) as FileLeaseRecord; - } catch (error) { - if (isErrorWithCode(error, "ENOENT") || error instanceof SyntaxError) - return undefined; - throw error; - } } +// Seeds and reads pass through JSON, so callers see what a persisted backend +// would return and cannot mutate the backend through a shared reference. function cloneAsPersistedState( state: Record, ): Record { diff --git a/packages/state/test/state-backend.test.ts b/packages/state/test/state-backend.test.ts index fde5e5b..ac1a198 100644 --- a/packages/state/test/state-backend.test.ts +++ b/packages/state/test/state-backend.test.ts @@ -1,25 +1,12 @@ -import { mkdtemp, rm } from "node:fs/promises"; -import { tmpdir } from "node:os"; -import path from "node:path"; import { describe, expect, it } from "vitest"; -import { - FileStateBackend, - MemoryStateBackend, - type StateBackend, - type StateNode, -} from "src/state"; - -type BackendFixture = { - backend: StateBackend; - cleanup: () => Promise; -}; +import { MemoryStateBackend, type StateNode } from "src/state"; function createStateNode( id: string, overrides: Partial = {}, ): StateNode { return { - rev: 0, + version: 1, id, groupId: 1, groupType: "stack", @@ -33,211 +20,43 @@ function createStateNode( }; } -function runStateBackendContractTests( - label: string, - createBackend: () => Promise, -) { - describe(label, () => { - it("starts with empty state", async () => { - const fixture = await createBackend(); - - try { - await expect(fixture.backend.get("missing")).resolves.toBeUndefined(); - await expect(fixture.backend.has("missing")).resolves.toBe(false); - await expect(fixture.backend.values()).resolves.toEqual([]); - } finally { - await fixture.cleanup(); - } - }); - - it("merges patches on update", async () => { - const fixture = await createBackend(); - const initialNode = createStateNode("resource-a"); - - try { - await fixture.backend.update(initialNode.id, 0, initialNode); - await fixture.backend.update(initialNode.id, 1, { - output: { status: "ready" }, - lastOperation: "update", - }); - - await expect(fixture.backend.get(initialNode.id)).resolves.toEqual({ - ...initialNode, - rev: 2, - output: { status: "ready" }, - lastOperation: "update", - }); - } finally { - await fixture.cleanup(); - } - }); - - it("rejects stale updates and deletes", async () => { - const fixture = await createBackend(); - const initialNode = createStateNode("resource-a"); - - try { - await expect( - fixture.backend.update(initialNode.id, 0, initialNode), - ).resolves.toEqual({ rev: 1 }); - await expect( - fixture.backend.update(initialNode.id, 0, { output: {} }), - ).rejects.toMatchObject({ - name: "RevConflict", - expectedRev: 0, - actualRev: 1, - }); - await expect( - fixture.backend.delete(initialNode.id, 0), - ).rejects.toMatchObject({ - name: "RevConflict", - }); - } finally { - await fixture.cleanup(); - } - }); - - it("treats expectedRev 0 as an expect-absent assertion", async () => { - const fixture = await createBackend(); - const initialNode = createStateNode("resource-a"); - - try { - await expect( - fixture.backend.update(initialNode.id, 0, initialNode), - ).resolves.toEqual({ rev: 1 }); - await expect( - fixture.backend.update(initialNode.id, 0, initialNode), - ).rejects.toMatchObject({ - name: "RevConflict", - expectedRev: 0, - actualRev: 1, - }); - } finally { - await fixture.cleanup(); - } - }); - - it("deletes values", async () => { - const fixture = await createBackend(); - const initialNode = createStateNode("resource-a"); - - try { - await fixture.backend.update(initialNode.id, 0, initialNode); - await fixture.backend.delete(initialNode.id, 1); - - await expect( - fixture.backend.get(initialNode.id), - ).resolves.toBeUndefined(); - await expect(fixture.backend.has(initialNode.id)).resolves.toBe(false); - await expect(fixture.backend.values()).resolves.toEqual([]); - } finally { - await fixture.cleanup(); - } - }); - - it("returns all values", async () => { - const fixture = await createBackend(); - const firstNode = createStateNode("resource-a"); - const secondNode = createStateNode("resource-b"); - - try { - await fixture.backend.update(firstNode.id, 0, firstNode); - await fixture.backend.update(secondNode.id, 0, secondNode); - - const values = await fixture.backend.values(); - - expect(values).toHaveLength(2); - expect(values).toEqual( - expect.arrayContaining([ - { ...firstNode, rev: 1 }, - { ...secondNode, rev: 1 }, - ]), - ); - } finally { - await fixture.cleanup(); - } - }); - - it("holds and renews an exclusive lease", async () => { - const fixture = await createBackend(); - - try { - const lease = await fixture.backend.lease("resource:a", 1_000); - const firstExpiry = lease.expiresAt; - await expect( - fixture.backend.lease("resource:a", 1_000), - ).rejects.toMatchObject({ name: "LeaseConflict" }); - - await lease.renew(2_000); - expect(lease.expiresAt).not.toBe(firstExpiry); - await lease.release(); +describe("MemoryStateBackend", () => { + it("starts with empty state", async () => { + const backend = new MemoryStateBackend(); - const next = await fixture.backend.lease("resource:a", 1_000); - await next.release(); - } finally { - await fixture.cleanup(); - } - }); + await expect(backend.get("missing")).resolves.toBeUndefined(); + await expect(backend.values()).resolves.toEqual([]); }); -} - -runStateBackendContractTests("FileStateBackend", async () => { - const tempDirectory = await mkdtemp(path.join(tmpdir(), "notation-state-")); - return { - backend: new FileStateBackend(path.join(tempDirectory, "state.json")), - cleanup: () => rm(tempDirectory, { recursive: true, force: true }), - }; -}); -runStateBackendContractTests("MemoryStateBackend", async () => ({ - backend: new MemoryStateBackend(), - cleanup: async () => undefined, -})); + it("returns seeded nodes by id", async () => { + const node = createStateNode("resource-a"); + const backend = new MemoryStateBackend({ [node.id]: node }); -describe("FileStateBackend", () => { - it("serialises concurrent CAS writers so only one wins", async () => { - const tempDirectory = await mkdtemp(path.join(tmpdir(), "notation-state-")); - const statePath = path.join(tempDirectory, "state.json"); - const first = new FileStateBackend(statePath); - const second = new FileStateBackend(statePath); - const initialNode = createStateNode("resource-a"); - - try { - await first.update(initialNode.id, 0, initialNode); - - const results = await Promise.allSettled([ - first.update(initialNode.id, 1, { output: { writer: "first" } }), - second.update(initialNode.id, 1, { output: { writer: "second" } }), - ]); - - const fulfilled = results.filter((r) => r.status === "fulfilled"); - const rejected = results.filter((r) => r.status === "rejected"); - expect(fulfilled).toHaveLength(1); - expect(rejected).toHaveLength(1); - expect((rejected[0] as PromiseRejectedResult).reason).toMatchObject({ - name: "RevConflict", - }); - await expect(first.get(initialNode.id)).resolves.toMatchObject({ - rev: 2, - }); - } finally { - await rm(tempDirectory, { recursive: true, force: true }); - } + await expect(backend.get(node.id)).resolves.toEqual(node); + await expect(backend.get("missing")).resolves.toBeUndefined(); }); -}); -describe("MemoryStateBackend", () => { it("returns values in deterministic id order", async () => { - const backend = new MemoryStateBackend(); const laterNode = createStateNode("resource-z"); const earlierNode = createStateNode("resource-a"); + const backend = new MemoryStateBackend({ + [laterNode.id]: laterNode, + [earlierNode.id]: earlierNode, + }); + + await expect(backend.values()).resolves.toEqual([earlierNode, laterNode]); + }); - await backend.update(laterNode.id, 0, laterNode); - await backend.update(earlierNode.id, 0, earlierNode); + it("isolates reads from the seed object and from each other", async () => { + const node = createStateNode("resource-a"); + const backend = new MemoryStateBackend({ [node.id]: node }); - await expect(backend.values()).resolves.toEqual([ - { ...earlierNode, rev: 1 }, - { ...laterNode, rev: 1 }, - ]); + node.output["name"] = "mutated-seed"; + const read = await backend.get(node.id); + read!.output["name"] = "mutated-read"; + + await expect(backend.get(node.id)).resolves.toMatchObject({ + output: { name: "resource-a-output" }, + }); }); }); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 3c1cc93..344636a 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -151,18 +151,21 @@ importers: examples/reconciler: dependencies: + '@notation/core': + specifier: workspace:* + version: link:../../packages/core '@notation/reconciler': specifier: workspace:* version: link:../../packages/reconciler '@notation/resource': specifier: workspace:* version: link:../../packages/resource - '@notation/state-sqlite': - specifier: workspace:* - version: link:../../packages/state-sqlite '@notation/utils': specifier: workspace:* version: link:../../packages/utils + yieldstar: + specifier: 0.5.0 + version: 0.5.0 devDependencies: '@types/node': specifier: ^22.13.4 @@ -261,9 +264,12 @@ importers: '@notation/state': specifier: workspace:* version: link:../state - '@notation/state-sqlite': - specifier: workspace:* - version: link:../state-sqlite + '@yieldstar/core': + specifier: 0.5.0 + version: 0.5.0 + '@yieldstar/sqlite-runtime': + specifier: 0.5.0 + version: 0.5.0 deep-object-diff: specifier: ^1.1.9 version: 1.1.9 @@ -276,6 +282,15 @@ importers: pako: specifier: ^2.1.0 version: 2.1.0 + pino: + specifier: ^9.14.0 + version: 9.14.0 + valibot: + specifier: ^1.4.2 + version: 1.4.2(typescript@6.0.3) + yieldstar: + specifier: 0.5.0 + version: 0.5.0 devDependencies: '@types/common-tags': specifier: ^1.8.4 @@ -379,30 +394,29 @@ importers: '@notation/state': specifier: workspace:* version: link:../state + '@yieldstar/core': + specifier: 0.5.0 + version: 0.5.0 deep-object-diff: specifier: ^1.1.9 version: 1.1.9 + valibot: + specifier: ^1.4.2 + version: 1.4.2(typescript@6.0.3) yieldstar: - specifier: ^0.4.6 - version: 0.4.6 + specifier: 0.5.0 + version: 0.5.0 + devDependencies: + '@yieldstar/sqlite-runtime': + specifier: 0.5.0 + version: 0.5.0 + pino: + specifier: ^9.9.0 + version: 9.14.0 packages/resource: {} packages/state: - dependencies: - '@notation/utils': - specifier: workspace:* - version: link:../utils - devDependencies: - '@types/node': - specifier: ^22.13.4 - version: 22.13.4 - - packages/state-sqlite: - dependencies: - '@notation/state': - specifier: workspace:* - version: link:../state devDependencies: '@types/node': specifier: ^22.13.4 @@ -2720,8 +2734,12 @@ packages: cpu: [x64] os: [win32] - '@yieldstar/core@0.4.6': - resolution: {integrity: sha512-6aJQ2NwA07YKdQpiyxdg+SCKwnHQsuxAGGW6uKZYydTS10ljL3uTE6mKdrt/22ehOtbXzjRa/xnECAWh3GM7pw==} + '@yieldstar/core@0.5.0': + resolution: {integrity: sha512-KaN1+AVg54W9G4VXHNmCRixV0MY123714YZgaA9EZx7mDgB0vfFA6T71rUnldGH86QEkocCwI4GCZzZ+0G4fyQ==} + + '@yieldstar/sqlite-runtime@0.5.0': + resolution: {integrity: sha512-zdG4kvOzEJFrTW52SY+WtdUuSvKhXs438H2/+TQotwi3fltLqva9EMgRJ/l9mcvC7nqAq1i0g2v+TZWjO/py4A==} + engines: {node: '>=22.6'} abstract-logging@2.0.1: resolution: {integrity: sha512-2BjRTZxTPvheOvGbBslFSYOUkr+SjPtOnrLP33f+VIWLzezQpZcqVg7ja3L4dBXmzzgwT+a029jRx5PCi3JuiA==} @@ -3719,19 +3737,9 @@ packages: pino-abstract-transport@2.0.0: resolution: {integrity: sha512-F63x5tizV6WCh4R6RHyi2Ml+M70DNRXt/+HANowMflpgGFMAym/VKm6G7ZOQRjqN7XbGxK1Lg9t6ZrtzOaivMw==} - pino-abstract-transport@3.0.0: - resolution: {integrity: sha512-wlfUczU+n7Hy/Ha5j9a/gZNy7We5+cXp8YL+X+PG8S0KXxw7n/JXA3c46Y0zQznIJ83URJiwy7Lh56WLokNuxg==} - - pino-std-serializers@7.0.0: - resolution: {integrity: sha512-e906FRY0+tV27iq4juKzSYPbUj2do2X2JX4EzSca1631EB2QJQUqGbDuERal7LCtOpxl6x3+nvo9NPZcmjkiFA==} - pino-std-serializers@7.1.0: resolution: {integrity: sha512-BndPH67/JxGExRgiX1dX0w1FvZck5Wa4aal9198SrRhZjH3GxKQUKIBnYJTdj2HDN3UQAS06HlfcSbQj2OHmaw==} - pino@10.3.1: - resolution: {integrity: sha512-r34yH/GlQpKZbU1BvFFqOjhISRo1MNx1tWYsYvmj6KIRHSPMT2+yHOEb1SG6NMvRoHRF0a07kCOox/9yakl1vg==} - hasBin: true - pino@9.14.0: resolution: {integrity: sha512-8OEwKp5juEvb/MjpIc4hjqfgCNysrS94RIOMXYvpYCdm/jglrKEiAYmiumbmGhCvs+IcInsphYDFwqrjr7398w==} hasBin: true @@ -3882,9 +3890,6 @@ packages: resolution: {integrity: sha512-57frrGM/OCTLqLOAh0mhVA9VBMHd+9U7Zb2THMGdBUoZVOtGbJzjxsYGDJ3A9AYYCP4hn6y1TVbaOfzWtm5GFg==} engines: {node: '>= 12.13.0'} - real-require@1.0.0: - resolution: {integrity: sha512-P4nbQYQfePJxRSmY+v/KINxVucm4NF3p3s7pJveMTtom52FR4YGltUQLB8idDXwDDWW+eYrWDFbuzUnjoWHF7g==} - recast@0.23.12: resolution: {integrity: sha512-dEWRjcINDu/F4l2dYx57ugBtD7HV9KXESyxhzw/MqWLeglJrsjJKqACPyUPg+6AF8mIgm+Zi0dZ3ACoIg+QtpA==} engines: {node: '>= 4'} @@ -4027,9 +4032,6 @@ packages: resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==} engines: {node: '>=8'} - sonic-boom@4.2.0: - resolution: {integrity: sha512-INb7TM37/mAcsGmc9hyyI6+QR3rR1zVRu36B0NeGXKnOOLiZOfER5SA+N7X7k3yUYRzLWafduTDvJAfDswwEww==} - sonic-boom@4.2.1: resolution: {integrity: sha512-w6AxtubXa2wTXAUsZMMWERrsIRAdrK0Sc+FUytWvYAhBJLyuI4llrMIC1DtlNSdI99EI86KZum2MMq3EAZlF9Q==} @@ -4118,10 +4120,6 @@ packages: thread-stream@3.1.0: resolution: {integrity: sha512-OqyPZ9u96VohAyMfJykzmivOrY2wfMSf3C5TtFJVgN+Hm6aj+voFhlK+kZEIv2FBh1X6Xp3DlnCOfEQ3B2J86A==} - thread-stream@4.2.0: - resolution: {integrity: sha512-e2zZ96wSChazBsbENf/Pcm/4swHt2cEKQ92rhUjkL9GCKiTDJIaTBenjE/m9DXi0QBmTMDkFDdOomUy20A1tDQ==} - engines: {node: '>=20'} - tiny-invariant@1.3.3: resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==} @@ -4344,6 +4342,18 @@ packages: util-deprecate@1.0.2: resolution: {integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==} + uuid@11.1.1: + resolution: {integrity: sha512-vIYxrBCC/N/K+Js3qSN88go7kIfNPssr/hHCesKCQNAjmgvYS2oqr69kIufEG+O4+PfezOH4EbIeHCfFov8ZgQ==} + hasBin: true + + valibot@1.4.2: + resolution: {integrity: sha512-gjdCvJ6d3RyHAneqxMYMW9QMCwYMb3jpOO0IyHZV1bnRHFBHrX3VkIILt5XYR0WhwHiH7Mty8ovuPZ/O3gamrg==} + peerDependencies: + typescript: '>=5' + peerDependenciesMeta: + typescript: + optional: true + vfile-message@4.0.3: resolution: {integrity: sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw==} @@ -4524,8 +4534,8 @@ packages: yallist@3.1.1: resolution: {integrity: sha512-a4UGQaWPH59mOXUYnAG2ewncQS4i4F43Tv3JoAM+s2VDAmS9NsK8GpDMLrCHPksFT7h3K6TOoUNn2pb7RoXx4g==} - yieldstar@0.4.6: - resolution: {integrity: sha512-toVxX0hHx+AlD5S3MT7voOW/Jw7AX5Mqejp28BJErSHGdZVcuKyVIfzYknbfnRnmBxAeZd3wfVnk7JQJQzZUZQ==} + yieldstar@0.5.0: + resolution: {integrity: sha512-MKuo2uaHYdy+1u0O0Q3Po3eVLGEmiNACsbaQ9aqOD4miZyRT63fXc/mhk0V4n8fNtoMN1BORD9QiXq5ZuOBtkQ==} yoctocolors@2.1.1: resolution: {integrity: sha512-GQHQqAopRhwU8Kt1DDM8NjibDXHC8eoh1erhGAJPEyveY9qqVeXvVikNKrDz69sHowPMorbPUrH/mx8c50eiBQ==} @@ -6657,7 +6667,15 @@ snapshots: '@voidzero-dev/vite-plus-win32-x64-msvc@0.1.16': optional: true - '@yieldstar/core@0.4.6': {} + '@yieldstar/core@0.5.0': + dependencies: + '@standard-schema/spec': 1.1.0 + + '@yieldstar/sqlite-runtime@0.5.0': + dependencies: + '@yieldstar/core': 0.5.0 + pino: 9.14.0 + uuid: 11.1.1 abstract-logging@2.0.1: {} @@ -7116,7 +7134,7 @@ snapshots: fast-json-stringify: 7.0.0 find-my-way: 9.6.0 light-my-request: 6.6.0 - pino: 10.3.1 + pino: 9.14.0 process-warning: 5.0.0 rfdc: 1.4.1 secure-json-parse: 4.1.0 @@ -7689,40 +7707,20 @@ snapshots: dependencies: split2: 4.2.0 - pino-abstract-transport@3.0.0: - dependencies: - split2: 4.2.0 - - pino-std-serializers@7.0.0: {} - pino-std-serializers@7.1.0: {} - pino@10.3.1: - dependencies: - '@pinojs/redact': 0.4.0 - atomic-sleep: 1.0.0 - on-exit-leak-free: 2.1.2 - pino-abstract-transport: 3.0.0 - pino-std-serializers: 7.1.0 - process-warning: 5.0.0 - quick-format-unescaped: 4.0.4 - real-require: 0.2.0 - safe-stable-stringify: 2.5.0 - sonic-boom: 4.2.1 - thread-stream: 4.2.0 - pino@9.14.0: dependencies: '@pinojs/redact': 0.4.0 atomic-sleep: 1.0.0 on-exit-leak-free: 2.1.2 pino-abstract-transport: 2.0.0 - pino-std-serializers: 7.0.0 + pino-std-serializers: 7.1.0 process-warning: 5.0.0 quick-format-unescaped: 4.0.4 real-require: 0.2.0 safe-stable-stringify: 2.5.0 - sonic-boom: 4.2.0 + sonic-boom: 4.2.1 thread-stream: 3.1.0 pirates@4.0.7: {} @@ -7838,8 +7836,6 @@ snapshots: real-require@0.2.0: {} - real-require@1.0.0: {} - recast@0.23.12: dependencies: ast-types: 0.16.1 @@ -8026,10 +8022,6 @@ snapshots: slash@3.0.0: {} - sonic-boom@4.2.0: - dependencies: - atomic-sleep: 1.0.0 - sonic-boom@4.2.1: dependencies: atomic-sleep: 1.0.0 @@ -8102,10 +8094,6 @@ snapshots: dependencies: real-require: 0.2.0 - thread-stream@4.2.0: - dependencies: - real-require: 1.0.0 - tiny-invariant@1.3.3: {} tinybench@2.9.0: {} @@ -8310,6 +8298,12 @@ snapshots: util-deprecate@1.0.2: {} + uuid@11.1.1: {} + + valibot@1.4.2(typescript@6.0.3): + optionalDependencies: + typescript: 6.0.3 + vfile-message@4.0.3: dependencies: '@types/unist': 3.0.3 @@ -8487,9 +8481,9 @@ snapshots: yallist@3.1.1: {} - yieldstar@0.4.6: + yieldstar@0.5.0: dependencies: - '@yieldstar/core': 0.4.6 + '@yieldstar/core': 0.5.0 nanoid: 5.1.16 pino: 9.14.0 serialize-error: 11.0.3