Uh oh!
There was an error while loading. Please reload this page.
[APPS-2792] Add: harden the in-process local execution path - #480
Conversation
This comment has been minimized.
This comment has been minimized.
6e85225 to
64c7a61Compare64c7a61 to
41a772eCompare59e9b78 to
6a19936CompareSerializes local backend-function executions via a promise-chain queue, since @datadog/action-catalog and @datadog/apps-backend both register runtime context via a shared, module-level setter that isn't safe under concurrent in-process execution. Also populates $.Source with a synthetic local-dev identity (deferred from Milestone 0), and adds edge-case coverage: non-serializable results, a top-level module throw, and a real concurrent-execution test against a genuine @datadog/apps-backend typed import confirming no cross-execution state leakage.
Runs the readOwnArgsAfterDelay concurrency check through the real, serialized executeScriptLocally entrypoint (its test.skip counterpart against PR #479's un-serialized base fails with cross-contaminated args). Passing here confirms the enqueue/queueTail promise-chain mutex actually closes the globalThis.$ race, not just reorders interleaved work.
An abandoned (timed-out) execution can keep running in the background after enqueue() lets the next execution start. Its late executeAction call previously ran for real, under whatever execution's identity happened to be current — with no error at all. Two distinct paths needed separate guards. A raw $.Actions call made through a reference captured before abandonment closes over its own execution's executeAction, so a simple abandoned flag inside that closure is enough to reject it. An @datadog/action-catalog typed-wrapper call is structurally different: action-catalog holds exactly one executeAction implementation in shared, module-level state, and always invokes whichever is currently registered — a per-closure guard is unreachable once a newer execution's registration overwrites it. That path needs the registration itself proactively replaced with a rejecting stub as soon as an execution concludes: unconditionally on normal completion (enqueue's mutex guarantees nothing newer has started yet), and on timeout only if nothing newer has started (checked via a per-execution epoch counter, to avoid clobbering a newer execution's own valid registration).
Code review of the epoch-gate/poisoning mechanism found three real gaps: - run()'s finally block poisoned the action-catalog registration unconditionally, contradicting its own "always safe" comment — when an abandoned execution's fn() settles late (after losing the timeout race), this could overwrite a newer, currently-valid execution's registration. Both the finally block and the timeout handler now share one epoch-gated concludeExecution() helper. - The initial registration calls themselves (not just the later poison step) were unguarded: a slow-to-resolve registration from an abandoned execution could land after a newer execution had already registered. run() now re-poisons immediately if it discovers it was abandoned while those calls were still in flight. - registerBackendRuntimeIfInstalled (apps-backend's setBackend) had no poisoning at all, despite this file's own comment naming it as sharing the identical shared-module-level-setter hazard as action-catalog's setExecuteActionImplementation. It now mirrors the same register/poison pattern. Also wraps both poison calls in try/catch — an external package's setter throwing on re-registration must not be allowed to mask a successful result via finally, or abort the timeout's setTimeout callback before it can reject.
6a19936 to
24c072fCompare
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Pull request overview
Friend, this PR hardens in-process backend execution with serialization, stale-context guards, and JSON-result validation.
Changes:
- Serializes local executions and poisons concluded runtime registrations.
- Validates returned values for JSON serialization.
- Expands concurrency, timeout, registration, and result tests.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
packages/plugins/apps/src/vite/local-execution.ts | Adds execution hardening and result validation. |
packages/plugins/apps/src/vite/local-execution.test.ts | Adds hardening regression coverage. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
tyffical
commented
Aug 21, 2026
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit:24c072fd57
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
- runScriptLocally now checks abandoned right before invoking the customer function, not just after the initial registration Promise.all — a slow loadModule call could otherwise still let an already-known-abandoned execution's function run pointlessly. - The finally block poisoning the shared registrations now covers loadModule and export validation too, not just the fn() call itself — a rejection from either of those previously skipped concludeExecution() entirely, leaving the registrations live. - assertJsonSerializable now returns the JSON-round-tripped value instead of the original — dev-server.ts serializes the result again for the HTTP response, so returning the original ran any custom toJSON()/getters a second time, which can throw or produce a different value than what was just validated.
…ed poisoning registerActionCatalogIfInstalled/registerBackendRuntimeIfInstalled's loadModule call is async and can be slow enough for a newer execution to start and validly register its own implementation while the older one is still in flight. Once it resolves, the older execution's initial register() call fired unconditionally, overwriting the newer execution's valid registration with its own stale one — and poisoning afterward can't undo that, since poisoning only replaces whatever's currently installed, not restore what was there before. Moves the epoch-currency check inside register() itself so every call (the initial registration and any later poison call) re-validates freshness at the moment it actually runs, instead of trusting a check done earlier. Separately, the backend-runtime poison object (a Proxy that throws on any property access) was being routed back through buildRuntimeFromJsFunctionWithActions, which synchronously reads and validates $.Source before producing a runtime — so the poison's own throw fired during that validation, before setBackend was ever reached, silently swallowed by the best-effort catch and leaving the previous, stale runtime installed. Poisoning now calls setBackend directly with an already runtime-shaped object whose accessors reject when actually invoked, instead of routing a broken $ back through the validating factory.
tyffical
commented
Aug 21, 2026
@codex review |
💡 Codex Reviewbuild-plugins/packages/plugins/apps/src/vite/local-execution.ts Lines 186 to 188 in b0cb069  Preserve poisoning across the next typed registrationWhen execution A times out, ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
packages/plugins/apps/src/vite/local-execution.ts:189
- This assertion trusts the shape of an external
unknownrequest, contrary to the repository's type-safety rule againstascasts. Narrow the object andconnectionIdexplicitly before callingnextExecuteAction; this also prevents a malformed runtime request from passing a non-string connection ID through a string-typed API.
setExecuteActionImplementation(async (actionId: string, request: unknown) => {
const { inputs, connectionId } = (request ?? {}) as Partial<ActionCallArgs>;
return nextExecuteAction(actionId, inputs, connectionId);
});
packages/plugins/apps/src/vite/local-execution.ts:464
- The cleanup handles are assigned only after both registration promises settle. If one SDK probe registers successfully while the other hangs, the timeout calls
concludeExecution()while both handles are stillundefined; if the hanging probe never settles, the successful module-level registration is never poisoned. Likewise, a rejection after the sibling has registered skips the later cleanup block. Capture and poison each registration independently as soon as it settles, and ensure registration-phase failures also run conclusion cleanup.
[reRegisterActionCatalog, poisonBackendRuntime] = await Promise.all([
registerActionCatalogIfInstalled(loadModule, guardedExecuteAction, isCurrent),
registerBackendRuntimeIfInstalled(loadModule, $, isCurrent, func.name),
]);
Motivation
@datadog/action-catalog'ssetExecuteActionImplementationand@datadog/apps-backend'ssetBackendboth register runtime context via a shared, module-level setter. Without serialization, a second concurrent execution's registration could silently redirect the first's still-in-flight typed-import calls to the wrong identity — with no error at all. See the RFC's Decisions and Trade-Offs.executeActioncall could still run for real, attributed to whichever execution was current by then. That needed a second, independent mechanism (see Changes below) beyond the queue itself.Changes
enqueue) — never run concurrently. A rejected execution doesn't wedge the queue for whatever's next.BigIntgets a clear, attributed error instead of an opaque downstreamJSON.stringifyfailure; a bare function/Symbol(whichJSON.stringifysilently drops instead of throwing) is also caught explicitly.executeActioncalls are now rejected instead of silently running under a newer execution's identity. Two distinct call paths needed separate guards: a raw$.Actionscall made through a reference captured before abandonment is caught by a per-closureabandonedflag; an@datadog/action-catalogtyped-wrapper call is structurally different — it always invokes whichever implementation is currently registered in shared module-level state, so a per-closure guard is unreachable once a newer execution's registration overwrites it. That path is closed by proactively replacing the registration with a rejecting stub as soon as an execution concludes: unconditionally on normal completion (the queue's mutex guarantees nothing newer has started yet), and on timeout only if nothing newer has started yet (checked via a per-execution epoch counter, to avoid clobbering a newer execution's own valid registration).globalThisorder marker, not a mock); the queue keeps flowing after an earlier execution rejects; aloadModulerejection (simulating a native-module load failure) rejects cleanly; all three non-serializable-result shapes; the no-token-exposure and$.Sourceinvariants from #479 are re-verified against the queued path; an abandoned execution's captured$.Actionsreference rejects even after a newer execution has taken over; an abandoned execution's action-catalog typed-wrapper call is rejected rather than silently running under a newer registration.QA Instructions
yarn test:unit packages/plugins/apps/src/vite/local-execution.test.ts # Expected: Test Suites: 1 passed / Tests: 23 passed ✅ VERIFIEDyarn test:unit packages/plugins/apps # Expected: Test Suites: 24 passed / Tests: 326 passed ✅ VERIFIEDyarn workspace @dd/apps-plugin run typecheck # Expected: no output, clean exit ✅ VERIFIEDnpx eslint packages/plugins/apps/src/vite/local-execution.ts packages/plugins/apps/src/vite/local-execution.test.ts --quiet # Expected: no output, clean exit ✅ VERIFIEDManual QA — real scaffolded app, real dev server, real timeout
This module isn't independently reachable from
npm run devon its own (that requires #481) — exercised via a real scaffolded app running the full stack (npm link'd@datadog/vite-pluginbuilt from this stack's tip):Added a backend function that captures
$.Actionsup front, sleeps 15s (past the 10s default timeout), then attempts a real$.Actions.foo.bar(...)call:{"success":false,"error":"Local execution of \"hangThenCallAction\" timed out after 10000ms"}Confirmed via the dev server's own log that the abandoned call, ~5s later, was rejected immediately with
"...was abandoned after timing out; refusing to run \"com.datadoghq.foo.bar\"..."— no real HTTP call to Datadog's API went out. ✅ VERIFIEDNote for anyone repeating this: the first attempt at this test showed the call going out for real (a genuine
preview-asyncrequest reachingapi.datadoghq.com, rejected only by the server'sACTION_NOT_FOUND, not by this fix) — traced to a stalenpm link'd build that hadn't picked up this branch's latest commit (prepare-linkhad linked an olddist/). Forcingrm -rf dist && yarn build:all-no-typesbefore re-linking resolved it and reproduced the expected rejection. Worth flagging since it's an easy false negative to chase for anyone else QA-ing this branch after a rebase.Blast Radius
local-execution.tsstill isn't called from anywhere in the existing dev server.Out of Scope / Follow-ups
handleExecuteAction, threading a realLoadModule,/__dd/executeActionViaCloudsplit, realpreview-asynccalls)$.Actionsexecutionnet.Socket.prototype.connect,fetch, andchild_process'sspawn/exec/execSyncfor the duration of a local execution, exempted only around the internal$.Actions→executeActioncallAsyncLocalStorage-based scoping, which the RFC already defers as needing upstream changes to both@datadog/action-catalogand@datadog/apps-backendDocumentation