📄 Say that XMD scans for credentials by default, and prove it over each backend - #575
Merged
Conversation
…ch backend The journal-gates page still described secret detection as something a host brings: a `containsCredential` placeholder inside `guardDurableStream`. Every execution has scanned by default since #199, so the page taught the wrong common path, and the CLI reference did not mention the opt-out at all. The page now leads with the built-in policy — default-on offline scanning installed before the first live event, what a finding does to the offending append and to the run, output withholding, the diagnostic that names a rule and never a value, and replay that does not rescan. `secretDetection: false` and `--no-secret-detection` appear as a dangerous diagnostic escape hatch with the warning the CLI actually writes, not as remediation. `guardDurableStream` keeps its gate-ordering, per-event and replay explanation, now as the generic decorator it is, and its example refuses an oversized event rather than deciding what a credential is. The reference lists the option with the same invocation-wide semantics. Two regressions cover the real default scanner over the two backends the memory cases could not reach. Each puts a synthetic, run-time-assembled credential in the root source, passes the backend straight to `execute()` with default options, and asserts `Err(SecretDetectedError)`, the absence of the rejected import, an independently safe `Close(err)`, and the canary nowhere in the persisted events or the error's readable fields — through the JSONL bytes on disk for the file backend, and through a read-back for HTTP.
PR #575: 📄 Say that XMD scans for credentials by default, and prove it over each backend3 files, +215 / -24 Scope✅ PR scope looks good. Structural✅ No structural bloat detected. Slop✅ Slop indicators look low. Static Analysis✅ Oxlint found no issues. CorrectnessNo extraneous code patterns detected. |
taras
marked this pull request as ready for review
August 24, 2026 13:58
Uh oh!
There was an error while loading. Please reload this page.
This was referenced Aug 24, 2026
taras added a commit
that referenced
this pull request
Aug 25, 2026
Six exact claims outlived the capabilities that disproved them, and one of them was damage I introduced while fixing another. `<Fetch>` no longer owns the pull-request evidence: the specification said "that read is `<Fetch>`'s" and "no stage writes those fetches yet" three sections away from the stage that now writes all three URL-addressed reads. Default-on secret rejection was still listed open in the primitives matrix after #573 and #575 shipped it. And one sentence read "That gate is and that gate is default-on" — a duplicated fragment from an earlier edit, which is exactly the kind of repair that should be made in place rather than appended later. The larger correction is what #301 slice 2 *is*. Four passages called it "scheduling the loop and continuing it unattended", which is the opposite of the boundary: slice 2 is the composition — the authored loop from discovery through acceptance under one run — and scheduling, watchers and unattended resume are deliberately outside it, with the later scheduling slice belonging to #300. Two more presented running the stages by hand under `xmd run` as the current execution path; it is history, and the composed root runs under `xmd workflow start`. The sweeps missed every one of these while exiting 0, which is the real defect. They now carry the nine exact defective sentences from the previous head as forbidden forms, self-tested, and fail against that revision while passing here. A scan for repeated phrases guards the duplication class directly, since a sweep that only knows the claims it was taught cannot see prose damage. The planning record is collapsed to match: the A/B/C sequence is gone from the composition plan, the implementor handoff and issue #301. A shipped in #577, B shipped in #580, and PR #181 owns C. The twelve-criterion matrix is unchanged — only its ownership column moves, with criteria 2 and 2a inherited from #577 and criterion 6's provider and replay halves from #580.
taras added a commit
that referenced
this pull request
Aug 25, 2026
Six exact claims outlived the capabilities that disproved them, and one of them was damage I introduced while fixing another. `<Fetch>` no longer owns the pull-request evidence: the specification said "that read is `<Fetch>`'s" and "no stage writes those fetches yet" three sections away from the stage that now writes all three URL-addressed reads. Default-on secret rejection was still listed open in the primitives matrix after #573 and #575 shipped it. And one sentence read "That gate is and that gate is default-on" — a duplicated fragment from an earlier edit, which is exactly the kind of repair that should be made in place rather than appended later. The larger correction is what #301 slice 2 *is*. Four passages called it "scheduling the loop and continuing it unattended", which is the opposite of the boundary: slice 2 is the composition — the authored loop from discovery through acceptance under one run — and scheduling, watchers and unattended resume are deliberately outside it, with the later scheduling slice belonging to #300. Two more presented running the stages by hand under `xmd run` as the current execution path; it is history, and the composed root runs under `xmd workflow start`. The sweeps missed every one of these while exiting 0, which is the real defect. They now carry the nine exact defective sentences from the previous head as forbidden forms, self-tested, and fail against that revision while passing here. A scan for repeated phrases guards the duplication class directly, since a sweep that only knows the claims it was taught cannot see prose damage. The planning record is collapsed to match: the A/B/C sequence is gone from the composition plan, the implementor handoff and issue #301. A shipped in #577, B shipped in #580, and PR #181 owns C. The twelve-criterion matrix is unchanged — only its ownership column moves, with criteria 2 and 2a inherited from #577 and criterion 6's provider and replay halves from #580.
taras added a commit
that referenced
this pull request
Aug 26, 2026
Six exact claims outlived the capabilities that disproved them, and one of them was damage I introduced while fixing another. `<Fetch>` no longer owns the pull-request evidence: the specification said "that read is `<Fetch>`'s" and "no stage writes those fetches yet" three sections away from the stage that now writes all three URL-addressed reads. Default-on secret rejection was still listed open in the primitives matrix after #573 and #575 shipped it. And one sentence read "That gate is and that gate is default-on" — a duplicated fragment from an earlier edit, which is exactly the kind of repair that should be made in place rather than appended later. The larger correction is what #301 slice 2 *is*. Four passages called it "scheduling the loop and continuing it unattended", which is the opposite of the boundary: slice 2 is the composition — the authored loop from discovery through acceptance under one run — and scheduling, watchers and unattended resume are deliberately outside it, with the later scheduling slice belonging to #300. Two more presented running the stages by hand under `xmd run` as the current execution path; it is history, and the composed root runs under `xmd workflow start`. The sweeps missed every one of these while exiting 0, which is the real defect. They now carry the nine exact defective sentences from the previous head as forbidden forms, self-tested, and fail against that revision while passing here. A scan for repeated phrases guards the duplication class directly, since a sweep that only knows the claims it was taught cannot see prose damage. The planning record is collapsed to match: the A/B/C sequence is gone from the composition plan, the implementor handoff and issue #301. A shipped in #577, B shipped in #580, and PR #181 owns C. The twelve-criterion matrix is unchanged — only its ownership column moves, with criteria 2 and 2a inherited from #577 and criterion 6's provider and replay halves from #580.
taras added a commit
that referenced
this pull request
Aug 26, 2026
Six exact claims outlived the capabilities that disproved them, and one of them was damage I introduced while fixing another. `<Fetch>` no longer owns the pull-request evidence: the specification said "that read is `<Fetch>`'s" and "no stage writes those fetches yet" three sections away from the stage that now writes all three URL-addressed reads. Default-on secret rejection was still listed open in the primitives matrix after #573 and #575 shipped it. And one sentence read "That gate is and that gate is default-on" — a duplicated fragment from an earlier edit, which is exactly the kind of repair that should be made in place rather than appended later. The larger correction is what #301 slice 2 *is*. Four passages called it "scheduling the loop and continuing it unattended", which is the opposite of the boundary: slice 2 is the composition — the authored loop from discovery through acceptance under one run — and scheduling, watchers and unattended resume are deliberately outside it, with the later scheduling slice belonging to #300. Two more presented running the stages by hand under `xmd run` as the current execution path; it is history, and the composed root runs under `xmd workflow start`. The sweeps missed every one of these while exiting 0, which is the real defect. They now carry the nine exact defective sentences from the previous head as forbidden forms, self-tested, and fail against that revision while passing here. A scan for repeated phrases guards the duplication class directly, since a sweep that only knows the claims it was taught cannot see prose damage. The planning record is collapsed to match: the A/B/C sequence is gone from the composition plan, the implementor handoff and issue #301. A shipped in #577, B shipped in #580, and PR #181 owns C. The twelve-criterion matrix is unchanged — only its ownership column moves, with criteria 2 and 2a inherited from #577 and criterion 6's provider and replay halves from #580.
taras added a commit
that referenced
this pull request
Aug 26, 2026
Six exact claims outlived the capabilities that disproved them, and one of them was damage I introduced while fixing another. `<Fetch>` no longer owns the pull-request evidence: the specification said "that read is `<Fetch>`'s" and "no stage writes those fetches yet" three sections away from the stage that now writes all three URL-addressed reads. Default-on secret rejection was still listed open in the primitives matrix after #573 and #575 shipped it. And one sentence read "That gate is and that gate is default-on" — a duplicated fragment from an earlier edit, which is exactly the kind of repair that should be made in place rather than appended later. The larger correction is what #301 slice 2 *is*. Four passages called it "scheduling the loop and continuing it unattended", which is the opposite of the boundary: slice 2 is the composition — the authored loop from discovery through acceptance under one run — and scheduling, watchers and unattended resume are deliberately outside it, with the later scheduling slice belonging to #300. Two more presented running the stages by hand under `xmd run` as the current execution path; it is history, and the composed root runs under `xmd workflow start`. The sweeps missed every one of these while exiting 0, which is the real defect. They now carry the nine exact defective sentences from the previous head as forbidden forms, self-tested, and fail against that revision while passing here. A scan for repeated phrases guards the duplication class directly, since a sweep that only knows the claims it was taught cannot see prose damage. The planning record is collapsed to match: the A/B/C sequence is gone from the composition plan, the implementor handoff and issue #301. A shipped in #577, B shipped in #580, and PR #181 owns C. The twelve-criterion matrix is unchanged — only its ownership column moves, with criteria 2 and 2a inherited from #577 and criterion 6's provider and replay halves from #580.
taras added a commit
that referenced
this pull request
Aug 26, 2026
Six exact claims outlived the capabilities that disproved them, and one of them was damage I introduced while fixing another. `<Fetch>` no longer owns the pull-request evidence: the specification said "that read is `<Fetch>`'s" and "no stage writes those fetches yet" three sections away from the stage that now writes all three URL-addressed reads. Default-on secret rejection was still listed open in the primitives matrix after #573 and #575 shipped it. And one sentence read "That gate is and that gate is default-on" — a duplicated fragment from an earlier edit, which is exactly the kind of repair that should be made in place rather than appended later. The larger correction is what #301 slice 2 *is*. Four passages called it "scheduling the loop and continuing it unattended", which is the opposite of the boundary: slice 2 is the composition — the authored loop from discovery through acceptance under one run — and scheduling, watchers and unattended resume are deliberately outside it, with the later scheduling slice belonging to #300. Two more presented running the stages by hand under `xmd run` as the current execution path; it is history, and the composed root runs under `xmd workflow start`. The sweeps missed every one of these while exiting 0, which is the real defect. They now carry the nine exact defective sentences from the previous head as forbidden forms, self-tested, and fail against that revision while passing here. A scan for repeated phrases guards the duplication class directly, since a sweep that only knows the claims it was taught cannot see prose damage. The planning record is collapsed to match: the A/B/C sequence is gone from the composition plan, the implementor handoff and issue #301. A shipped in #577, B shipped in #580, and PR #181 owns C. The twelve-criterion matrix is unchanged — only its ownership column moves, with criteria 2 and 2a inherited from #577 and criterion 6's provider and replay halves from #580.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Closes#199. Secret detection has been on by default since slice 2, but the
website still taught it as something a host brings, and the CLI reference did
not mention the opt-out at all. Anyone reading the docs to decide whether XMD
protects their journal came away with the wrong answer.
The test corpus had the matching gap: the real default scanner was only ever
proven over
InMemoryStream. Every file and HTTP case installed atest-supplied gate instead, so nothing showed that
execute()'s own scannerreaches those two backends.
What changes
Before:
/docs/journalopened withguardDurableStreamand acaller-supplied
containsCredentialplaceholder, and told the reader that"deciding what counts as a secret is your call, not this library's."
/docs/referencelisted five CLI options and no way to turn detection off.After:
/docs/journalopens with the built-in policy and states theobservable contract — default-on offline scanning installed before the first
live event; what a finding does to the offending append and to the run; output
withholding; diagnostics that name a rule and never a value; replay that does
not rescan; and the host-only opt-out as a dangerous diagnostic escape hatch.
guardDurableStreamfollows it as the generic decorator it is./docs/referencelists--no-secret-detectionwith the same invocation-widesemantics.
No production code changed. Behavior is exactly what shipped in #329, #330 and
#573.
Review guide
Start with:
site/routes/docs/journal.tsxThen review:
site/routes/docs/reference.tsx— the new CLI option entrypackages/core/tests/guarded-journal.test.ts— the newdescribe("the default scanner over a persistence backend")blockLook carefully at:
execute(). If either onegrows a
guardDurableStreamwrapper, it stops testing the thing it exists totest.
Close(err)rather than an empty backend.What must stay true
the rendered page;
site/deno task checkanddeno task buildkeep itcompiling and formatted.
--no-secret-detectionstays the only spelling that disables detection.Enforced by
packages/cli/src/cli.tsand checked bypackages/cli/tests/secret-detection-cli.test.ts; the docs now state it.offending event is absent and that
close(root)with statuserrispresent.
time from an alphabet slice; nothing reads an environment variable, Git
credential, or user configuration.
How to verify it
keeps a credential out of a file backendproves the real default scannerrejects a credential before file persistence, and sweeps the JSONL bytes on
disk. It fails if the scanner is not installed, if the rejected import reaches
the backend, or if a partial record is written.
keeps a credential out of an HTTP backendproves the same before HTTPpersistence, reading back through the stream rather than anything the run
retained.
messageandstackare canary-free, so a diagnosticthat quoted the matched value would fail them.
Mutation evidence. On a throwaway copy of the file with the two new
execute()calls changed tosecretDetection: falseand nothing else touched,both new cases failed at
expect(result.ok).toBe(false)while the five existingcases still passed. They discriminate the default rather than an incidental
refusal.
Commands run at
74ef1d7d, after onedeno task setupin a fresh worktree:deno task test packages/core/tests/secret-detection.test.ts packages/core/tests/guarded-journal.test.tsok | 3 passed (51 steps) | 0 failedpnpm exec tsx --tsconfig tsconfig.node.json --test --test-concurrency=1 <same two files>tests 43 / pass 43 / fail 0bun test --timeout=300000 <same two files>43 pass / 0 failbun test --timeout=300000 -t "keeps a credential out of" packages/core/tests/guarded-journal.test.ts2 pass / 5 filtered out / 0 fail / 18 expect() callssite/deno task checksite/deno task build✓ built in 1.38sdeno task lintdeno task checkdeno task check:jsrSuccess Dry run completeThe fourth row is not redundant.
bun testprints only counts, so a suite thatsilently skipped the two new cases would also report
43 pass; the filtered runis what shows they ran under Bun and that their assertions executed.
Scope
Included
/docs/journalrewrite and the/docs/referenceCLI option entry.Intentionally unchanged
durable event shape, output bridging, settlement, replay, or persistence
adapter is touched.
specs/executable-mdx-spec.md§8.1already states this contract; the website now matches it.
ci.ymlstaysas it is and remains the authority for that surface.
secret-detection.test.tsalready proves diagnostics, output withholding, settlement, replay and the
disabled control. Those are execution-owned behavior, not backend variants, so
each backend gets one focused case.
scripts/runtime-test-exclusions.tsand run under all three runtimes.built-in policy, so the label under-describes it, but renaming would touch
site/routes/docs/_layout.tsxandsite/routes/docs/agents.tsx— out of scopehere, and flagged rather than done quietly.
Risks and limitations
pages were rendered from the built site and read as an operator would see them;
every claim traces to
specs/executable-mdx-spec.md§8.1 or topackages/cli/src/cli.ts, including the warning text quoted verbatim.Scope confirmation