Automated dogfood campaign in the Dev Container - #503
Open
IanFrelinger wants to merge 45 commits into
Open
Conversation
…open criticals DO NOT MERGE. This branch is a work-in-progress record of a dogfooding exercise, and it ends on four unresolved critical findings. Detail below. Eight agents built real applications on Ashlar as outside developers, consuming it only through NuGet packages, with the rule that reading framework source to make something work is itself a finding. All eight shipped. They filed 104 problems; 40 after dedupe; the top 12 each survived two independent skeptics. WHAT IS FIXED AND VERIFIED - The mutation leg no longer signs a vacuous verdict. A brick with no namespace got an audit context that never loaded, so every mutant threw, every throw counted as a kill, and escape_rate came out 0 on a witness with no teeth. - A multi-file brick is no longer certified against one arbitrarily-chosen file. - The instance ledger detects tail truncation via a signed head anchor stored outside the ledger directory, with a two-phase write so a crash between the entry and the anchor cannot brick the project. - The analyzer fence actually reaches consumers: the package ships under analyzers/dotnet/cs, and the dependency leg no longer accepts a bare PrivateAssets="all", which does not keep the analyzer out of the built brick. - A brick referencing Ashlar.Authoring certifies instead of dying on CS0009: the loader no longer feeds unmanaged native DLLs to Roslyn, and a compile failure that is entirely CS0009 is now a harness error naming the loader. - verify, run, self-extend and doctor stopped reporting success for work that did not happen. export refuses rather than shipping a bundle whose own launcher exits 65. - Three documentation pages that did not exist: the certification gate (with the first definition anywhere of "witness"), consuming from NuGet, and the operator lifecycle. 14 false "not on nuget.org" claims corrected. cert-gate 441/441. Infrastructure 2116 (net8) and 2231 (net10), 0 failures. CLI 239/239, Kernel 456/456, BackgroundAgents 542/542. Package-only restore-build-run still works first try on both TFMs. WHY THIS MUST NOT MERGE — four critical bypasses, one root cause The gate decides WHICH FILES it is certifying by globbing *.cs and parsing the csproj as XML, rather than asking MSBuild what it compiles. Every heuristic patch moved the hole: - <Compile Include="x.cstxt"/> is compiled by csc and invisible to the hash, the analyzer fence and the mutation leg. Two bricks, different behaviour, identical contentHash, both ADMIT. Its [ModuleInitializer] ran inside the certification harness. - Sub/obj/Payload.cs: the SDK excludes only ROOT obj/ and bin/; the gate excludes those names at any depth. No csproj edit required. - Directory.Build.props beside the csproj bypasses both checkers, which XDocument.Load a single file while MSBuild evaluates the whole chain. - <Compile Remove> plus a non-.cs include makes the hashed set disjoint from the compiled set, so the record is signed over a decoy. The fix is to derive the certified source set from the evaluated project. That needs a deliberate decision about evaluating attacker-influenced csproj files, which is why it is not attempted here. Also open: the "history was deleted" ledger refusal names a reanchor command that refuses an empty ledger; export is unfixable for a project whose policy narrows sandbox.root; and a bounded daemon reports faulted on an ordinary SIGTERM because its operator-stop guard never receives a cancellation token. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… progress Checkpoint so parallel worktrees can branch from a known state. Infrastructure builds clean; not verified end to end. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The assigned defect — RestoreOnlyFix naming `ashlar ledger reanchor` for the
entries-gone state while ReanchorAsync refused an empty ledger — was already
closed in HEAD by AcceptDestroyedHistoryLocked. I re-ran the reported repro
against a live build before touching anything: certify 2 entries,
`rm -rf .ashlar/ledger`, then status (65) -> the named fix (exit 0) ->
status (0) -> verify (0). The dead end is gone.
So I ran the audit the brief actually asks for: provoke every refusal in
InstanceLedgerAnchor.cs and LedgerCommand.cs, then RUN the fix each one names.
All eleven named fixes execute and clear the state they are attached to. What
they do not all do is tell the truth, and both defects were in the places with
the most confident test coverage.
1. The success line called a destruction marker a survivor.
`reanchor --yes` printed "{Count} surviving entr(y|ies)" for both outcomes.
Where every entry was gone, Count is 1 and that 1 is the marker the call had
just written — so an operator who lost their entire history was told
"OK ledger re-anchored - 1 surviving entry". The number was right and the
sentence was false, in the success message of the recovery path this whole
pass exists to fix. The two outcomes are not one sentence with a different
number in it: the entries-gone case now reports "NOTHING survived ... this is
not a recovery. nothing was recovered." The truncation control still reports
its real survivors.
The distinction is carried on LedgerVerification.DestroyedHistoryAccepted
rather than inferred by the caller, because inferring it means sniffing the
refusal text and a rewording would quietly restore the lie.
2. The no-key refusal named a directory it had not searched, and a fix that
looped forever.
It printed OperatorKey.ResolveKeyDir() regardless of --key-dir, so
`--key-dir /somewhere/empty` sent the operator to ~/.ashlar/keys. Worse, it
said to run bare `ashlar keys init` and "then run this command again" — and
bare keys init writes to that same default, which this command does not
search. Verified live: run the named fix verbatim, re-run the command, get
the byte-identical refusal. Forever. That is the exact failure the brief says
has now shipped twice, in the one refusal whose test asserted only that the
substring "ashlar keys init" appeared.
It now names the directory it really searched and emits
`ashlar keys init --key-dir "<that dir>"`. Confirmed live in both branches:
copy-paste the command, re-run, exit 0, ledger verifies.
3. The permanent record of a loss repeated an unattested claim as fact.
The entries-gone state is refused before the anchor's signature is examined,
in the read path and the append path alike. That ordering is deliberate and I
left it: checking the signature first would route an operator whose entries
are gone AND whose anchor is unsigned to `ashlar verify`, which refuses that
same state — a new dead end one state over, which is how this defect class
keeps regenerating. There is now a comment saying so, so the next round does
not "fix" it into one. What changed is only honesty: an unsigned anchor's
sequence and hash are recorded as its claim, found on disk, not as a length
anything attested. An anchor carrying no head hash no longer writes a
dangling "(hash )" into a signed, permanent entry.
Also renamed Reanchor_refuses_to_pin_an_empty_ledger. It pins the empty-and-no-
anchor case, but its name reads as "empty ledgers are refused" — the belief that
caused the original defect, sitting in the test suite waiting to justify its
return.
docs/OperatorLifecycle.md taught the working command already; it now shows what
the entries-gone outcome actually prints, states the unattested-claim rule, and
documents the --key-dir trap.
Verification: 467/467 kernel, 254/254 CLI, plus the full refusal audit re-run
against the rebuilt binary. Every fix named by every refusal in these two files
was executed, not read.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>…ance `ashlar export` was an unfixable refusal for any project whose policy narrowed sandbox.root, and the refusal named a fix that could not be executed. The staging half of that landed in the previous WIP commit (StageApp now creates the relative directories the policy names). This closes the rest of the class, reproduced live on both `export native` and `export aws`. W3b — a RELATIVE sandbox.root that resolves OUTSIDE the project (`../shared`) is the same defect one shape further out, and it survived the first fix. The source project certifies (the directory is right there) and the root can never travel, so the export refused — and answered with "make it relative to the project" about a root that was already relative, and "a RELATIVE directory is created inside the bundle for you" about a directory it had just declined to create. Every bullet was false for the failure in hand, so re-running reproduced the refusal byte for byte. The fix list is now built from what actually failed: this case names `mkdir -p <project>/workspace` and `root: ./workspace`, and those steps were run verbatim — export, self-verification and the aws bundle all then pass. W3c — an ABSOLUTE sandbox.root ships a silent 65. It names a directory on a machine, so the staged copy verifies wherever the exporting machine happens to have it: export exits 0, bundle.json says certified, and the launcher's first line (`verify --path app`) then fails the envelope course on the machine the bundle is handed to. That is exactly what the self-verification exists to prevent and the one case it structurally cannot see. Refusing would be wrong — deploying onto a machine you provision is legitimate, and a refusal with no way past it is the defect this pass is about — so the export prints a portability note naming both real fixes. Both were run: moving the root into the project makes the bundle verify on a machine without the original directory; provisioning the directory makes the exported bundle verify as-is. Also, every other refusal on the export path, each with its named fix run verbatim against the failure it claims to fix: - the symlink refusal printed the path relative to whichever subtree was being copied, so `rm Widget/link.cs` for a link at src/Widget/link.cs — it names the full path now, and deleting it exports cleanly. - "not an ashlar project" named no next step; it now names `ashlar init <name>` and `--path`, matching `ashlar verify`'s wording. - "the project does not verify" withheld courses this command had already computed and sent the operator to a second command to be told them; the failing courses are printed inline (BundleInfo carries them). Staging directories on the operator's behalf is only defensible said out loud, so the export now discloses exactly which directories it created and why — on the console beside the verdict and in the bundle README — and creates nothing else: a writable path that escapes the root is left for the envelope course to refuse at the source rather than quietly materialised into a policy the verifier accepts. `./out/` and `./out` also stopped being counted as two directories. Verified live in the dev container: narrowed, escaping, absolute, nested, trailing-slash and dot-dot roots; native + aws + --zip; the zip round-trips the empty directories and the extracted bundle verifies; the cloud bundle really builds and its container's verify-then-run entrypoint exits 0, with a negative control (the directory removed from the image) exiting 65. An ordinary scaffolded project and a brick outside src/ still export and self-verify. 257/257 CLI tests. Not mine, found on the way: ProjectVerifier.VerifyEnvelope reads `writable: - .` as escaping a `sandbox.root` written with a trailing slash. It fails closed at the source with a named course, so it is a verify defect, not an export one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… MSBuild says
Four certification bypasses shared one root cause: the gate decided which files
it was certifying by globbing *.cs and reading the .csproj as XML, instead of
asking what the project actually compiles. Four rounds of more careful
name-matching each moved the hole rather than closing it.
The evaluated-compile-set work at HEAD (an unreviewed, unverified first cut)
closes those four by asking MSBuild instead of modelling it. This finishes it,
and closes three more that were found while verifying it — all reproduced live
on this repo, each with the payload's type present in the built assembly:
5. <Target BeforeTargets="CoreCompile"> adding a Compile item contributes
nothing at evaluation time and everything at compile time.
6. Give that target a Condition on $(OutputPath) and it contributes nothing
to a SECOND MSBuild query either — the gate builds into a temp directory
and a separate query does not. "Verify with another invocation", which is
what HEAD did, splits in exactly the same way as the four before it.
7. Ask in the build's own invocation and a target AfterTargets="CoreCompile"
simply removes the payload from @(Compile) again. The compile has already
happened; every later reading of the item list is clean.
WHY THE SHAPE CHANGED. @(Compile) is mutable state belonging to the project
under judgement, and its targets run before the gate can read it. No reading of
it is an authority on what was compiled. The authority is the compiler's own
record — the source-document table csc writes into the PDB while emitting, from
the syntax trees it actually parsed — which is bound to the assembly by id and
cannot be edited after the fact without changing the assembly too. So:
* The certified set is still MSBuild's EVALUATED Compile list (no targets
run), because the gate must be able to refuse before the candidate's code
exists as a binary. That is the text the content hash covers.
* The build is now ONE `dotnet msbuild -restore -t:Build` invocation that also
reports its items, so there is no second property context to diverge from.
(`dotnet build -getItem:Compile` reports EVALUATION-time items even though
it runs targets — verified against the SDK in the dev container.)
* The verdict compares the hashed set against the PDB's document table, before
Assembly.LoadFrom, which is what executes a payload's module initializers.
* MSBuild's post-build answer is demoted to narrowing ONE tolerance: a
compiled file outside the hash is admitted only under the project's own
IntermediateOutputPath AND when MSBuild reports an SDK-shipped .targets file
declared it. Used that way its failure mode is a refusal, not an admission.
* DebugType=portable, ChecksumAlgorithm=SHA256, empty PathMap and
DeterministicSourcePaths=false are forced as GLOBAL properties, which the
project cannot override; no readable record left is then a refusal, not a
gap. A build that emits no PDB is refused.
* Each hashed file's captured BYTES must match the compiler's checksum for it,
so a target that rewrites the brick source on its way to CoreCompile can no
longer make the certificate describe a program the assembly does not
contain. The first cut of this compared against the file re-read from disk,
which passes on exactly the case it exists for; the test caught it.
Also fixed here: BaseIntermediateOutputPath comes back from MSBuild as the
literal "obj\" on Linux for some query shapes, so Path.Combine produced a
directory named `obj\`, every containment test against it failed, and the SDK's
own assembly-info stopped being recognised — a blanket refusal of every honest
brick, latent at HEAD.
REFUSAL MESSAGES. Three named a fix that leads into a DIFFERENT refusal
("declare it as an ordinary Compile item" → straight into the multi-file
refusal, because a brick is one source file). They now name a fix an author can
actually carry out, and a test walks one end to end. Every fix named in a
message in these files was executed and confirmed to work.
VERIFICATION. All five new regression tests were confirmed RED against the
pre-fix shape by temporarily reverting the authority (PDB → MSBuild items, one
invocation → two) and re-running, then restored. Bypasses 1/2/4 were confirmed
red against the original glob by running that exact algorithm against the
fixtures. cert-gate is 467/467 (was 441/441); Ashlar.Tests.Kernel 462/462;
Ashlar.Infrastructure builds clean on net8.0 and net10.0.
NOT BLANKET REFUSAL. Explicitly covered and green: an ordinary single-file
brick, SDK-generated assembly-info/global-usings under obj/, a project that
relocates its own intermediate output, a deterministic CI build, and the
analyzer ExcludeAssets shape docs/CertificationGate.md teaches — each asserted
to reach Assembly.LoadFrom rather than merely "not throw".
KNOWN RESIDUAL, stated in CompiledSourceDocuments.cs rather than left implied:
the gate reads the compiler's record out of the artifact. A target that forges
BOTH the assembly and a matching PDB would satisfy every check here. The honest
claim is "no file reaches the compilation without being hashed", not "the
assembly provably contains only the hashed source"; closing that needs the gate
to compile the hashed source itself and compare the IL.
BrickSourceResolver keeps the anchored bin/obj rule rather than refusing: it is
handed an application directory (zero or many .csproj), and `ashlar verify` runs
inside exported bundles on machines with no SDK, so there is no MSBuild to ask.
Its over-inclusion is the safe direction for an inventory and is documented as
such at the top of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>…pped A bounded `ashlar background-agent daemon --duration N` reported ok:false / status:faulted / reason:host_stopped_early, exited 1, and wrote "faulted" into the heartbeat the container HEALTHCHECK reads — on an ORDINARY SIGTERM. An honest stop reported as a failure makes a working deployment look broken, which is the more damaging direction of the defect this pass has been chasing. The immediate cause was known: the command inferred "the host stopped itself" from ApplicationStopping, and ConsoleLifetime raises exactly that on a `docker stop`. The guard meant to tell an operator's stop apart (cancellationToken.ThrowIfCancellationRequested) was dead code, because the handler called RunAsync with no token. Closing the reported instance is not enough, because the obvious repair re-arms it. "Faulted = ApplicationStopping fired AND something logged an error with an exception" reads DaemonFaultLog.HasFault, which ANY error line carrying an exception sets — and BackgroundAgentService logs exactly that, with LogError, for a single agent it could not create, then carries on working. One bad entry in an agent config, or one unreachable mesh peer over three weeks, and the next `docker stop` reports status:faulted and exits 1 again. So the class is closed rather than the instance: * A stop verdict is now a function of FOUR facts, ordered so each rung covers the one below it: a hosted service whose ExecuteTask actually faulted (structural — nothing else sets it, and a service cancelled by an ordinary shutdown ends cancelled, not faulted); then a stop signal positively observed (PosixSignalRegistration for SIGTERM/SIGINT/SIGQUIT, plus the command's token, because neither alone covers both `docker stop` and Ctrl+C); then the old logged-error reading; then a bare ApplicationStopping, which is a clean stop. * The bounded and unbounded paths now run the SAME judgement. They disagreed before — one signal, two runs, opposite verdicts — which is the defect itself. Only the consequence differs: bounded exits, unbounded parks. * The observed signal is reported (stop report "signal" field), so the mechanism stays checkable instead of asserted. Six rounds have ended on a claim nobody could check. Two further honesty defects found while verifying this one: * A faulting bounded run exited 1 naming the service and left "status":"running" in the heartbeat — the timer's first beat (it fires immediately) raced the fault report and won. The HEALTHCHECK reads that file and nothing else, so a node whose daemon had just died reported healthy. The heartbeat is now silenced (DisposeAsync, which waits for a beat in flight) before any terminal status is written. Verified faulted in three consecutive runs. * host.StopAsync throwing after a window that fully elapsed escaped to the park handler, which announced a daemon that "failed to start" and then retried for ever, never returning to the operator watching it. It now has its own report (reason "shutdown_failed") that says the run finished and the stop did not. The duration wait now completes through a TaskCompletionSource created with RunContinuationsAsynchronously. StopApplication runs its ApplicationStopping callbacks on the signal-handler thread; if the wait ever resumed inline there, shutdown would end in host.Dispose() disposing ConsoleLifetime's signal registrations, and PosixSignalRegistration.Dispose waits for its own handler to return. Not a bug observed here — one that cannot happen once resumption is guaranteed asynchronous, which is why WaitForShutdownAsync is built this way. DaemonStopReportTests exercised only the pure report builders, which is why two successive fixes stayed green over a rule that read its inputs wrongly. It now also tests WHERE THE FACTS COME FROM against a real HostBuilder: ApplicationStopping after StopApplication, a BackgroundService that threw (found from the service, with an EMPTY fault log), one merely cancelled by an ordinary shutdown, and the re-arm case above. Eight of the thirty-three fail against the previous rule — checked by putting it back, not assumed. Measured against the built CLI, not reasoned about: bounded+SIGTERM/SIGINT/SIGQUIT, unbounded+SIGTERM/SIGINT -> exit 0, reason "shutdown", signal named, stop within 1s, no faulted heartbeat window elapsing -> exit 0, reason "duration_elapsed", signal null agent config that faults the service, bounded -> exit 1, names BackgroundAgentService and "Agent broken-agent must have a name", cycles 0, heartbeat "faulted" (3/3 runs) same config, unbounded -> parks naming the same service, heartbeat "parked" parked node + SIGINT/SIGTERM -> stops promptly, never needs SIGKILL Every fix named in a message this touches was run: `ashlar background-agent report` exits 0, ASHLAR_LOG_JSON=1 produces the structured log including the Error line that names the failure, and the park message's "Ctrl+C, or `docker stop`" both stop a parked node. One measurement trap is documented in OperatorStopSignal for whoever debugs this next: a daemon started as a background job by a shell WITHOUT job control inherits SIGINT/SIGQUIT as SIG_IGN, and .NET honours that, so the daemon appears to ignore Ctrl+C entirely. It is the shell, not the daemon — SIGTERM still works, and the same binary stops instantly under `set -m`. That artefact cost most of the verification of this change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…not the output folder
No brick that referenced an Ashlar package could pass the analyzer leg unless
its author added <CopyLocalLockFileAssemblies>true</...>. CollectReferences
globbed *.dll out of the build output directory, the SDK does not copy package
assemblies into a library's output, so Roslyn was handed no assembly defining
Ashlar.Core.Domain.Bricks.Brick and the fence refused every stock brick with
"analyzer anchor type ... is not resolvable". Neither shipped sample nor the
`ashlar new brick` template set the property: the scaffold could not be
certified as scaffolded, and docs/CertificationGate.md papered over it with a
"things that will bite you" footnote.
The output directory was a model of the compiler's reference set, wrong in
the same direction the *.cs glob was a model of its source set. The loader now
uses the two records the build already produces:
- MSBuild's ReferencePathWithRefAssemblies items, requested in the SAME build
invocation EvaluatedBrickProject.Build already runs (-getItem), say where
the assemblies CoreCompile handed csc live - package assemblies in the NuGet
cache, no copy-local needed. Under a target-less evaluation MSBuild reports
the item as [], so Evaluate() is unaffected.
- The compiler's own metadata-reference record in the portable PDB
(CompiledMetadataReferences, new: file name + MVID per reference) decides
membership. A reported path is accepted for a recorded reference only when
the file there has the recorded MVID. MSBuild's post-build list is thereby
a path oracle, never an authority - the same stance CompiledSourceDocuments
takes for the source set.
Fail closed throughout: an empty reported list, a PDB without the record, or
a recorded reference with no matching file is a refusal naming the assembly
and the target shape to remove (verified: the fix each message names clears
it). The runtimes/<rid>/native and PEReader.HasMetadata filters are kept and
applied to the reported list, so the LLamaSharp CS0009 crash cannot return.
One deliberate difference from "what csc saw": the targeting pack's reference
assemblies are verified but withheld. RoslynCodeAnalysisService.BuildReferenceSet
already supplies the host runtime's framework, and a second System.Runtime
gave Roslyn two core libraries and CS0518 on every predefined type - observed
on the first run of the fix.
Reference resolution also moved BEFORE Assembly.LoadFrom, so a tampered build
is refused before the candidate's module initializers run.
Template: the scaffold also wrote an output key ("implementation") its
BrickInterface never declared, which the analyzer leg refuses (ASHLAR0002);
it is now declared, and the scaffold certifies exactly as generated -
BrickCertificationProjectLoaderReferenceTests reads the real template files,
substitutes the tokens, builds it and runs all five legs. Doc: the
CopyLocalLockFileAssemblies guidance is gone, Shape A states plainly that the
candidate's build targets and module initializers execute in the certifying
process before any leg runs, and the determinism bullet now says what leg 4
and the analyzer catalog actually cover (DateTime.Now/DateTimeOffset.Now,
unseeded Random, mutable statics, constructor-only I/O) rather than "leg 4
will catch it".
All five end-to-end tests were run against the pre-fix loader and fail there
(no refusal thrown; reference set = the brick dll alone). Cert-gate filter:
479/479.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>…tead of crash, and pin it on disk The PDB-authoritative loader (c729c41) exposed that the two shipped sample bricks had only ever certified BECAUSE of the bypass it closed: samples/Directory.Build.props injected <Compile Include="../src/Ashlar.Compat/GlobalUsings.DomainBrick.cs" /> into HelloBrick and Ashlar.Certified.DamageResolver - a file compiled into the signed assembly from outside the brick directory, invisible to the old *.cs glob. The gate is right to refuse it; the samples were wrong. At the same time the refusal escaped tools/Ashlar.ExportCertifiedBrick as an unhandled exception (exit 134, stack trace, designed message buried seven frames down) on both samples, and nothing in cert-gate could see any of it because no test ever loaded a tracked on-disk sample. Fix 1 - the export tool refuses instead of crashing. Loader-stage refusals (InvalidOperationException / FileNotFoundException) are caught at the gate boundary and printed WITHOUT the stack trace, exit 4 - distinct from 2 (gate REJECT) and 3 (post-export verify). The witness default was one hard-coded file name (damage-resolver.witness.json), so every other brick stopped on a bare FileNotFoundException; it is now the single *.witness.json beside the csproj, with a named-fix refusal when there is not exactly one. The tool also stops clearing ASHLAR_CERT_NUGET_CONFIG: pack-certified-brick-reuse.sh exports it one line before invoking the tool, and the clear made that a no-op. Ashlar.CertifyBrick did not crash, but shared exit 1 between REJECT and "the gate never ran"; refusals are now 3 and unexpected errors 4, documented in its usage line. Fix 2 - both samples certify via the documented package-only path, with no gate change. The injection block is gone from samples/Directory.Build.props; the file stays (empty, with a comment saying why) because it is what shields samples/ from the repository-root props. DamageResolverBrick names its base type Ashlar.Core.Domain.Bricks.Brick in full (its namespace starts with Ashlar., so the short name hits the Ashlar.Brick namespace). HelloBrick drops its ProjectReference into src/, takes Ashlar.Brick.Contracts 0.1.1 from nuget.org, derives from Brick directly and gains hello-brick.witness.json covering both outputs. Neither brick can carry a global using alias inline: the certification legs compile the single source under CandidateSourceWrapper, which prepends its own `using DomainBrick = ...;`, so an inline `global using` is CS8915 and a duplicate alias is CS1537 - the alias has to go, not move. Both csproj files carry CopyLocalLockFileAssemblies=true with a comment: the analyzer leg compiles against the brick's own build output and a library build does not copy package DLLs there (loader limitation, being fixed at the root; drop once it lands). DamageResolver's Brick.Contracts moves 0.1.0 -> 0.1.1 so the brick restores from a plain checkout with no local feed - 0.1.0 never existed on nuget.org, so the on-disk test would have failed in CI while passing on any machine whose package cache happened to hold a local pack. The record is regenerated through scripts/pack-certified-brick-reuse.sh (7 mutants, 0 survivors, content hash over the LF bytes the index stores). Two pre-existing defects in the same sample's by-hand flow, found by running it: ProjectB never set ManagePackageVersionsCentrally=false, so its explicit Versions were NU1008 under the root Directory.Packages.props; and its unused Ashlar.Authoring reference dragged nuget.org's Ashlar.Core.Application 0.1.1 -> Ashlar.Certification.Contracts >= 0.1.1 over the local 0.1.0 (NU1605). The verifier must be the same code as the signer that produced the record, so the local pack stays and Authoring goes - from ProjectB and from the pack script, which packed it for no consumer. ProjectB now restores and prints TRUSTED finalDamage=40. Fix 3 - ShippedSampleCertificationTests drives BrickCertificationProjectLoader.LoadAsync + CertificationGate against the REAL tracked samples/hello-brick/HelloBrick and samples/certified-brick-reuse/Ashlar.Certified.DamageResolver directories and asserts ADMIT, signed, mutants > 0, survivors == 0. It sits in Tests.Certification, so CERT_GATE_FILTER selects it (verified with --list-tests; full run-cert-gate.sh 470/470). Verified to FAIL against the pre-fix samples (both facts, with the exact "outside the brick directory" refusal) and pass after. Not done here, flagged for the owning lanes: samples/README.md and docs/AuthoringBricks.md, docs/CertificationGate.md, docs/DocsIndex.md still describe hello-brick as a non-certifiable ProjectReference sample and the props file as supplying a DomainBrick alias. And the content-bound record hashes raw bytes, so a CRLF working tree (core.autocrlf=true on Windows) reports content-hash-mismatch against the LF record; `*.cs text eol=lf` for the sample would make it platform-stable. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he arithmetic
The mutation leg's whole job is proving the witness would notice if the brick's
logic were wrong. It could not do that for arithmetic: a brick computing
Math.Max(0, baseDamage - armor) and one computing Math.Max(0, baseDamage + armor)
-- contradictory programs -- BOTH certified "ADMIT escape_rate=0 mutants_killed=5"
against the same witness, because the catalog produced three string-literal
mutants, one int-literal mutant and one statement removal, and never touched the
operator. Reproduced before the change: both bricks ADMIT with exactly those five
mutants killed (ArithmeticMutantTeethTests failed for that reason pre-fix).
The catalog now carries the operator classes a real mutation tester has:
swap-arithmetic-op + <-> -, * <-> /, % -> *
swap-arithmetic-assign += <-> -=, *= <-> /=, %= -> *=
shift-relational-boundary < <-> <=, > <-> >= (the off-by-one class flip-binary-op cannot express)
swap-unary-op -x <-> +x, ++x <-> --x, x++ <-> x--
remove-logical-not !x -> x
Every one of them consults the semantic model before rewriting a token. The
engine scores a non-compiling mutant as KILLED, so a swap that ignored types
("a" + "b" -> "a" - "b", enum + 1 -> enum - 1, !(bool?) dropped inside an if)
would inflate the kill count of a signed certificate with mutants that were
dead on arrival -- exactly the vacuity the leg exists to prevent. The catalog
binds the candidate against the SAME reference set and the same injected usings
the certification compile uses (hoisted to a sibling tree as global usings so
candidate line numbers and ToSource() are untouched); the engine now passes its
compilation references through. An operand whose type cannot be resolved yields
NO operator mutant rather than a guess. Whole-expression constants, constant-zero
divisors and steps of loop-control variables (for/while/do conditions) are
skipped: the first two are compile errors after a swap, the last is a mutant that
never returns and would hang the in-process harness instead of being judged.
OperatorMutationCatalogTests compiles every operator mutant of five fixtures,
including an operator zoo covering ints, doubles, decimals, int?, char, enum,
string, DateTime/TimeSpan, bool and bool?.
Ids stay {kind}-{line} with the existing #2/#3 disambiguation, and the five
pre-existing ids for the reproducing brick are pinned verbatim so records
signed before this change stay reproducible. Strategy names name every kind
(a test now checks the engine's list against what the catalog emits -- it caught
a missed edit during this work).
Consequence handled here, not handed off: the boundary operator exposed that
every "strong" probe-brick witness in the suite was toothless at the zero-error
boundary -- errorCount > 0 mutated to >= 0 survived because no witness ever fed
a log without an ERROR line. Those tests went red; that is the gate doing its
job. The witnesses were STRENGTHENED (MutationProbeWitnesses.ZeroErrorCase, one
shared case across ten sites), not the mutant weakened. All suites green.
Timing on the simple bricks (dotnet test durations, same machine): the probe
brick's strong-witness certification 3 s before and 3 s after (+1 mutant); the
line-substring-counter engine run 2-3 s before and 2 s after (+2 mutants); the
reproducing brick 5 -> 6 mutants. The binding-only compilation adds tens of
milliseconds per certification; the leg is bounded by the per-kind cap of 4,
so worst case grows from 28 to 48 mutants on a brick that has every shape.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>… record, drop the stopgap, fix the docs The three lanes merged without conflict (disjoint file sets), but they interact: - Lane gate regenerated samples/certified-brick-reuse/.../certification-record.json before lane export's operator-class mutants existed. Against the merged tree the record was stale: the mutation leg now derives 10 mutants from DamageResolverBrick.cs (was 7) - swap-arithmetic-op on the `* / 100` line and on `raw - armor` - and the shipped witness already kills all three, so no witness case had to be added. Regenerated through scripts/pack-certified-brick-reuse.sh; the contentHash (rPxj86wx...) is unchanged, only the mutant list, timestamp and signature moved. ProjectB restores against the regenerated feed and prints `TRUSTED finalDamage=40`, exit 0. HelloBrick has no arithmetic: 4 mutants, 0 survivors, export tool exit 0. - Lane gate added CopyLocalLockFileAssemblies=true to both samples as a stopgap for the loader reading references from the build output; lane ledger made the loader read the compiler's own reference record instead. Removed the stopgap from both samples: both still certify through the export tool and through ShippedSampleCertificationTests (on net8.0, the cert-gate framework). - The tracked record binds a SHA-256 over the exact bytes of DamageResolverBrick.cs, but the repository only normalised it to LF in the index; a Windows checkout with core.autocrlf=true wrote CRLF and the record could not verify locally. .gitattributes now pins both sample brick sources to eol=lf (verified: rm + git checkout writes LF on this autocrlf=true machine). - Docs still described the pre-fix samples (hello-brick "ProjectReference, not certifiable"; CertificationGate.md "copy the complete project below rather than either sample"; samples/README describing the props-file alias injection; hello-brick README calling the stopgap temporary). Rewritten to the merged state. Every command in the rewritten text was executed in this round. - AnalyzerFenceGate's CS0009 harness refusal said the reference set comes "from the brick's build output"; it now comes from the compiler's record. Reworded; the test pinning the component name (BrickCompiledSourceSetTests) still passes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…x the stale sample text An adjudicated HIGH finding: the documentation audience cannot reproduce what it reads. docs/CertificationGate.md is written for a nuget.org consumer and describes, in the present tense, a loader that reads the compiler's own record of the build, a reference set resolved from ReferencePathWithRefAssemblies, and an Ashlar.Analyzers package that runs a fence under ExcludeAssets="runtime;compile". None of that is in the packages nuget.org has. Checked against the v0.1.1 tag: BrickCertificationProjectLoader takes the first *.cs in the directory and globs *.dll out of the build output; BrickDependencyChecker allows exactly two packages with no build-time-only exception; Ashlar.Analyzers 0.1.1 ships lib/netstandard2.0/ only; and the Ashlar.CLI 0.1.1 tool embeds the template from before the loader change, whose scaffold the current gate REJECTS. The reverse holds too: HelloBrick and the tracked template, which carry no CopyLocalLockFileAssemblies, REJECT at the analyzer leg under a 0.1.1 host. - VERSION 0.1.1 -> 0.1.2. Nothing publishes on a VERSION change: release.yml runs on a v*.*.* tag push or workflow_dispatch (and asserts the tag matches VERSION), release-nuget.yml is workflow_dispatch only, release-staging-on-label.yml dispatches release.yml on the release:staging PR label. No tag is created here. The tracked DamageResolver record's analyzer-gate configuration carries analyzerAssemblyVersion=0.1.1.0, stamped from VERSION, so the next regeneration will move that string and the signature (no test pins the tracked bytes; ProjectB verifies the record as signed). - CHANGELOG.md: the [Unreleased] section now says the next release is 0.1.2 and lists the gate work landing in it - compiler-record source authority, references from the build, the analyzer package layout, operator-class mutants, and the bounded mutant execution, compile-option parity and cap/scope removal landing in parallel lanes. - docs/CertificationGate.md: a callout at the top states exactly what a 0.1.1 consumer gets; every loader, fence and packaging claim 0.1.1 lacks is marked "since 0.1.2"; the Shape A package references and the Ashlar.Analyzers snippet read 0.1.2; the in-process legs compiling under the build's DefineConstants is stated as 0.1.2 behaviour, not present fact. - docs/AuthoringBricks.md, samples/hello-brick/README.md, samples/templates/brick/README.md: the same present-tense claims about the template certifying as scaffolded and about CopyLocalLockFileAssemblies being unnecessary, marked the same way. - samples/README.md: the templates row said the scaffold needs a local feed or a ProjectReference swap; Ashlar.Authoring has been on nuget.org since 0.1.1. The reuse row and samples/certified-brick-reuse/README.md no longer hard-code totalMutants (the README said 7; the record says 10) and point at certification-record.json instead, so the next catalog change cannot leave them stale again. - samples/certified-brick-reuse/ProjectB/Program.cs: the two positional arguments are not interchangeable (one is hashed, one is parsed as JSON); given in the wrong order the program died on an unhandled JsonException, exit 134. Parsing is now explicit: wrong count, wrong extension, missing file or unparseable record all print a usage line and exit 2. Not covered by an xunit test: this lane runs no dotnet, and the sample program is not referenced by any test project. The merge agent builds it. - Ashlar.sln: tools/Ashlar.CertifyBrick and tools/Ashlar.ExportCertifiedBrick added under a new "tools" solution folder with fresh GUIDs and the six Debug/Release x Any CPU/x64/x86 mappings the other projects carry, so a solution build compiles the tools the docs tell the reader to run. Edited as text; not built in this lane. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ls the witness never earned
Two adjudicated HIGH findings against AstMutationCatalog, each reproduced by a pair of
contradictory bricks that certified TOGETHER against one witness:
- MaxPerKind = 4 stopped every operator kind after its first four qualifying sites in document
order. A shipping brick with six arithmetic sites never had `+ surcharge - discount` mutated, so
cap-minus and cap-plus (differing only in that operator) both certified escape_rate=0 against a
witness with surcharge=0, discount=0, and the record carried no trace of the truncation. The cap
is gone - none per kind, none per site. A brick is one file and a mutant costs a fraction of a
second; a cap that has to be surfaced in the record to keep the record honest is not worth it.
- IsExecutionMethod mutated ExecuteAsync and private INSTANCE methods only. Arithmetic in a
private static, internal or public helper - or a property body, a constructor, a nested type -
was never mutated, so static-minus/static-plus/internal-minus all certified with every kill owed
to input keys and a non-compiling statement removal. Scope is now every member body of every
type in the candidate (methods, constructors and their initializers, accessors, expression-bodied
members, property and field initializers; local functions and lambdas ride along), whatever the
modifiers. The loop-control guard stays.
The skeptics also flagged two kinds of kill the witness never earned; both are removed at the
catalog (the engine is untouched):
- Non-compiling mutants are discarded, not emitted. The engine scores them as killed, but that
kill is the compiler's. Each mutant is re-bound in the same binding compilation the operator
family already uses (candidate parsed alone, the wrapper's usings hoisted as global usings, the
engine's reference set). When the candidate itself does not bind, nothing can be judged and
nothing is discarded.
- Lookup-key literals are not mutated: the index of an element access that is read, and an
argument that binds to a NON-VOID method's parameter named `key` (BrickInput.Get, TryGetValue,
ContainsKey, GetValueOrDefault). A mutated lookup key fails on every input whatever the witness
expects. Store keys (BrickOutput.Set, dict[k] = v) stay mutable: dropping a declared output is
observable only by a witness that asserts it. An invocation that cannot be bound falls back to
a short list of lookup member names so a blind run behaves the same on the common shapes.
Widening the scope exposed equivalent mutants on honest bricks; handled at the operator:
- Identity operands are not swapped (x * 1 <-> x / 1, x + 0 <-> x - 0, and the compound forms).
- A constructor statement assigning a field or property NOTHING in the candidate reads (Id, Name,
Version, Description, Interface = new BrickInterface { ... }) is out of scope: no member computes
anything from it, so no witness case could kill a mutant of it. This is a data-flow rule, not a
name list: a member the constructor writes and any body reads (`_rate = 5`, multiplied in
ExecuteAsync) stays in. Without it every literal of every brick's metadata block survived.
- The probe fixtures' ExtractErrorMessage carried an unreachable guard (`if (idx < 0)` on lines
the caller had already filtered by "ERROR") and a TrimStart that absorbed an off-by-one in
`idx + 5`: three equivalent mutants the moment helpers were in scope. Rewritten as one reachable
expression in all four fixtures (compiled and source, probe and nondeterministic), so every
mutant of the helper is observable and StrongWitness still admits.
Samples, certified with the export tool built from this tree, 0 survivors each: DamageResolver 6
mutants (was 10: the four input-key kills and the non-compiling remove-statement-42 are gone; the
output-key mutant mutate-string-literal-53, which the cap had cut off, is in), HelloBrick 2 (its
two output keys; the input key and the non-compiling removal are gone), and the brick template
scaffold (BrickCertificationProjectLoaderReferenceTests). The tracked DamageResolver record is
regenerated through the export tool; contentHash unchanged, only mutants, timestamp and signature
moved.
The adversarial oracles, run through that tool: cap-minus and cap-plus both REJECT at mutation
(escape 0.25; swap-arithmetic-op-40 and #2 named with the `- discount` edit); static-minus,
static-plus and internal-minus all REJECT at mutation (escape 0.67; swap-arithmetic-op-37 named).
All five ADMITTED before this change.
Tests: MutationCoverageTeethTests (new), two VacuousMutationLegTests facts (kills must be owed to
an expectation; a witness that adds nothing over the vacuous one is toothless and rejected), and
the OperatorMutationCatalogTests id pin now asserts the input-key and remove-statement ids are
GONE rather than present. 23 of them fail at 8a6e862 and pass here. Focused 38/38; the
regression filter 112/112; nine further certification classes 56/56; conventions 7/7.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>…ons, and record them Every in-process leg of the gate — the analyzer fence, the mutation catalog's binding compilation and every per-mutant compile — parsed the brick with Roslyn's DEFAULT options: no preprocessor symbols, unchecked arithmetic, the host compiler's language version, no global usings. The real build used the evaluated project. So two projects with a byte-identical Brick.cs, and therefore an identical signed contentHash, compiled DIFFERENT programs, and the gate judged the one that did not ship: - /tmp/fifthway/EX vs EY (FifthBrick.cs sha256 c8540ebc..., identical): EY's csproj adds ASHLAR_EVIL to DefineConstants, which compiles a File.WriteAllText into ExecuteAsync. The fence saw the guarded lines as disabled text, the mutation leg never mutated them, and both ADMITTED. - No csproj edit is needed. Microsoft.NET.Sdk defines NET, NET8_0, NETCOREAPP and the *_OR_GREATER family for every net8.0 project inside a target, so `#if NET8_0` reaches the same split from a completely stock project. - CheckForOverflowUnderflow=true: the witness leg ran the built (checked) assembly, but every mutant was compiled unchecked, wrapped where the shipped brick throws, and failed the overflow case — so a mutant the witness could not observe was "killed" by the wrong options and the gate signed escape_rate=0 for a witness with no teeth. - <Using Include="System.DateTime" Alias="Clock" /> lands in obj/.../GlobalUsings.g.cs, which the loader already tolerated as SDK boilerplate but the legs never compiled. A using alias beats a namespace import, so the build bound Clock.Now to the wall clock while the fence bound it to a deterministic Fake.Clock declared in the source, and the fence passed a nondeterministic brick. All four reproduced at 8a6e862: CompiledProgramParityTests was written first and run against the base build — every fact ADMITTED (EY, the #if NET8_0 twin, the alias brick, the checked brick). What changes: - BrickCompileOptions (Core.Application) is the one authoritative description of "which program": language version, sorted preprocessor symbols, CheckOverflow, Nullable, AllowUnsafe, and the global using directives the build compiled. CertificationRequest carries it; null means "no build to match" (hot-swap and generated candidates compile in-process only, so that compile IS the program) and reproduces the previous defaults byte for byte. - BrickCompilation (Infrastructure) is the single mapping to Roslyn: ParseOptions, CompilationOptions, and the global usings as a sibling syntax tree — the shape csc saw them in. AnalyzerFenceGate, AstMutationCatalog (parse + binding compilation) and both mutant compiles in BrickMutationEngine consume it through one internal RoslynCodeAnalysisService overload; the public CompileAsync is unchanged. Node selection in the catalog and mutant execution are untouched. A finding located in the global-usings tree maps to candidate line 0, like the wrapper's own text. - The loader derives the options from the COMPILER'S OWN RECORD, not from MSBuild: CompiledCompilationOptions reads the compilation-options block csc writes into every portable PDB (language-version, define, checked, nullable, unsafe) — the same authority CompiledSourceDocuments and CompiledMetadataReferences already rest on. MSBuild's DefineConstants would have been the wrong source three ways: the SDK appends the implicit framework symbols inside a target, a post-build property read is mutable state belonging to the project under judgement, and any list of "the symbols the SDK defines" is a model of the compiler rather than the compiler. The record needs no list; it is what csc was given. The global usings come from the SDK-generated documents the compiler recorded compiling (AssertCompiledSetIsExactlyTheCertifiedSet now hands those back), each file's bytes checked against the compiler's checksum before a directive is taken from it, and a generated file that has grown a declaration is refused. - Refusals, not defaults, in the loader and before any leg runs or anything executes: a PDB without the options block, a language version this gate's Roslyn (4.8) cannot parse, a nullable context it does not know, a project `DomainBrick` alias for anything other than Ashlar.Core.Domain.Bricks.Brick (the harness injects that alias into every compile; the same-type alias — the old samples' shape — is dropped as the harmless duplicate it is, since CS1537 would otherwise fail honest bricks). Where the options are bound in the record, and why not in contentHash: The record carries them as a signed `compile-options` input: Id is the canonical line (langVersion=12.0;checkOverflow=false;nullable=Enable;unsafe=false;symbols=...;globalUsings=...), so a reader sees which program was judged, and Hash is its digest. Inputs are already inside the v2 signed payload (TrustLoopRecordSchemaTests pins tampering with them), so no schema version bump and no change to any existing record's bytes or signature. Folding the options into contentHash was considered and rejected: contentHash is the identity every downstream verifier recomputes from the source text alone — CertificationTrustVerifier.Verify(record, source) in ProjectB and the export tool's post-export check, the hot-swap host's verify-at-load, and the revocation list keyed on it — and changing its definition would silently break all of them, or force every consumer to obtain the csproj to verify a record. A sibling under the signature is equally tamper-evident: a record for (source, options A) cannot be presented as covering (source, options B), because the input is signed and its absence from a new record is also detectable. The requirement — nobody can ship a program the gate did not judge — is met from both sides: the legs now judge the program the build compiled, and the record names that program under the signature. Verified against the oracles with tools/Ashlar.CertifyBrick built from this tree: EX ADMIT (escape_rate=0, 6 mutants killed); EY REJECT (mutation): escape_rate=0.14, survivor mutate-string-literal-38 on the backdoor's own path literal. Two honest notes. First, EY is caught by the mutation leg, not the fence: the shipped analyzer catalog has no file-IO rule outside constructors (BrickConstructorPurityAnalyzer), so File.WriteAllText in ExecuteAsync is not a fence finding; when the guarded code trips a fence rule (DateTime.Now, as in the alias fixture) the fence now rejects it. Second, because the fence passes EY, the correctness leg executed the built assembly in-process and the backdoor wrote /tmp/fifthway/out/twin-backdoor.txt before the mutation leg rejected — the same in-process execution the witness leg has always performed; containing it is the execution-backend lane's concern, not this one's. The tracked DamageResolver record is regenerated (export tool against the local feed, the pack-certified-brick-reuse.sh steps): contentHash rPxj86wx... and the 10 mutants are unchanged; the record gains the compile-options input, and its timestamp and signature move. ProjectB restores against the regenerated feed and prints `TRUSTED finalDamage=40`, exit 0. Tests (all [Trait Certification]; every one fails at 8a6e862): - CompiledProgramParityTests (real SDK builds through the loader, TestTimeouts.HostTouching): the EX/EY twin pair as inline fixtures (clean ADMITs, backdoor REJECTs, same contentHash, the record names ASHLAR_EVIL); the `#if NET8_0` twin with a stock csproj; the global-using-alias brick REJECTED at the fence with ASHLAR0006; the checked-arithmetic brick REJECTED at mutation with the unobserved string-literal survivor. Run at base first: all four ADMITTED. - BrickCompileOptionsTests (no SDK): the fence sees the guarded branch only under the symbol; the catalog derives more mutants under NET8_0 than a symbol-free parse (the negate-condition mutant exists only when the `if` is a node); the fence binds Clock through the build's alias; the engine reports the survivor under checked and vacuously kills it unchecked; the PDB reader round-trips a real Roslyn emit (C# 11, three symbols sorted, checked, nullable Enable, unsafe) and its defaults, and refuses an assembly with no record; an unknown language version and a foreign DomainBrick alias are refused by name; the canonical line is order-independent and is what the input signs. These use the new APIs and so fail to compile at base. Not closed here, noted for the attack phase: CodePage / non-UTF-8 source bytes (csc's fallback decoding vs the loader's UTF-8 read could differ inside string literals; the checksum is over bytes so it passes), <Features> and InterceptorsNamespaces (an interceptor in a single-file brick is visible to the fence and fails to compile in-process, so today's direction is fail-closed), and Optimize (IL differs, C# semantics do not). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ocess, and record who killed what
Two adjudicated blockers on the gate, one root cause: author code ran on the certifier's own
threads and nothing bounded it.
- CRITICAL. An honest brick with a plain countdown loop never finished certifying. The new
shift-relational-boundary mutant turns `while (n > 0) { n /= 10; }` into `while (n >= 0)`,
the witness case value=0 makes that nonterminating, and MutantWitnessExecutor invoked the
mutant's ExecuteAsync reflectively with no timeout. Oracle /tmp/adv-mut/fx/hang-gt through
tools/Ashlar.CertifyBrick at 8a6e862: killed by `timeout 90`, no verdict. (A pre-existing
variant, `while (n != 0)`, hangs the same way via flip-binary-op.)
- HIGH. An honest brick with a recursive private helper took the certifier down. A mutated
literal in `Dec(x) => x - 1` (or a flipped `<=` on the base case) makes the recursion infinite;
StackOverflowException cannot be caught, so the process died with exit 134, empty stdout, no
verdict, no message. Oracles /tmp/adv9/recur/RecurBrick and /tmp/skep-recur/DecBrick both
reproduce it at 8a6e862 in ~5s. The DecBrick variant needs no operator swap, so this class
predates the operator mutants.
A thread can be abandoned but not stopped, so an in-process "timeout" would leave a spinning
core behind for the life of the host — and Ashlar.Infrastructure also certifies inside the
autonomy loop's service process. A process can be killed. So:
- LocalProcessExecutionBackend (new) is the default ICandidateExecutionBackend when a request
names none. One child per job replays the candidate (repeats=2, serving the correctness AND
determinism legs) or every mutant sequentially, streaming one nonce-prefixed JSON line per
(unit, case, repeat) over stdout. The certifier enforces the wall clock: the runner times out
a case itself and then EXITS to shed the spinning thread; if the child hangs beyond that or
dies (exit 134, Environment.Exit, FailFast, background-thread throw, OOM abort), the certifier
kills it if needed, synthesises timed-out / crashed observations for the unit that was
executing, restarts the runner for the units that remain, and still returns a verdict. A runner
that cannot start, completes nothing, or exhausts the total budget throws
CertificationHarnessException — nothing was observed, so nothing is scored.
- WitnessReplayRunner (new) is that child: a ~200-line console program held as source and
compiled once per certification with the engine's own Roslyn service and the brick's own
reference set into the run's temp dir, next to a runtimeconfig.json for the runtime that is
running the certifier; launched through the dotnet muxer that launched this process (falling
back to DOTNET_HOST_PATH / DOTNET_ROOT / the running runtime's install root / PATH). No new
project, no package asset, no restore. Values cross the boundary TYPED ({"t":"i32","v":"5"})
in both directions, because BrickInput.Get<T> is an exact type check and the comparers
distinguish integral from floating values — a plain JSON number would have failed honest
bricks. The runner's own IExecutionContext carries AuditExecutionContext's values, so it needs
nothing from the candidate; the CandidateSourceWrapper still appends CertAuditContext because
the analyzer fence and the compiler must keep seeing one text.
- MutantWitnessExecutor and MutantAssemblyLoadContext are deleted; WitnessRunner.RunAsync and
CheckDeterminismAsync (the in-process legs) are gone with them. The mutation engine compiles
mutants here and executes them only through the backend. There is no in-process execution of
author code left in the gate. The engine no longer holds CollectibleLoadContextGate; the
hot-swap host keeps it for its own generations.
- The record is honest about who killed what. MutationTestResult and CertificationRecord carry
TimedOutMutants and CrashedMutants beside KilledMutants; TotalMutants is the sum of the four
lists, escape_rate is survivors/total, and a mutant the clock or a process death stopped is
never filed as a witness kill (crash outranks timeout outranks witness). Records are minted at
SchemaVersion 3 (CertificationRecordData.CurrentSchemaVersion); the v3 payload signs both new
lists, so editing a timed-out id into killedMutants does not verify. TrustLoopSchemaVersion
stays 2 as the floor constant and v2 records verify byte-for-byte as before. The per-run gate
passes record where execution happened and under what budget
(execution=local-process;perCaseTimeoutMs=5000;perUnitTimeoutMs=30000;totalTimeoutMs=600000;
heapLimitMb=1024), configurable through CertificationGate's new optional
CandidateExecutionLimits. Both tools print `mutants_killed=N killed_by_timeout=N[ids]
killed_by_crash=N[ids]` on the ADMIT/REJECT line; a mutation REJECT reason names them too.
- WitnessFindingKind gains TimedOut and Crashed, so a candidate that attacks its host is rejected
at correctness with a finding that says what happened rather than "threw"; RepairFeedback
renders both. A candidate the runner cannot LOAD is a harness refusal, not a correctness FAIL.
- The autonomy loop's identity-only handle (ProposedBrickHandle, ctor(string brickId)) is the one
brick shape without a parameterless constructor; the runner constructs it with the brick id so
its refusal still surfaces as Threw findings and RefusedInProcessExecution still recognises it.
Evidence, all in the dev container:
- BoundedCandidateExecutionTests (new, 10 facts). At 8a6e862, base-compatible variants of the
three headline facts were run one at a time: the looping-mutant fact hung until its 45s xunit
timeout; the recursive-helper fact aborted the run with "Test host process crashed : Stack
overflow."; the Environment.Exit(0) fact aborted it with "Test host process crashed". With this
change all ten pass in 16s.
- Oracles through tools/Ashlar.CertifyBrick, before -> after: hang-gt `timeout 90` (no verdict)
-> ADMIT in 9s, mutants=10 mutants_killed=9 killed_by_timeout=1[shift-relational-boundary-32];
RecurBrick exit 134 in 4s -> REJECT (mutation) in 5s, killed_by_crash=1, one survivor which is
the equivalent mutant `<= -> <` on the base case; DecBrick exit 134 in 5s -> REJECT (mutation)
in 5s, killed_by_crash=3[flip-binary-op-37, negate-condition-37, remove-statement-37], same
equivalent survivor. Neither recursion fixture is admissible as written, and both now say why.
- samples through tools/Ashlar.ExportCertifiedBrick, before -> after: hello-brick 4s -> 4s
(mutants=4, all witness kills); DamageResolver 5s -> 6s (mutants=10, all witness kills). The
tracked DamageResolver record is regenerated through scripts/pack-certified-brick-reuse.sh:
contentHash and mutant list unchanged, schemaVersion 3, the two new empty lists, and the
execution budget on its gate passes.
- Regression filter (ShippedSampleCertificationTests|BrickCertificationProjectLoader|
BrickCompiledSourceSetTests|BrickEvaluatedCompileSetTests|ArithmeticMutantTeethTests|
OperatorMutationCatalogTests|AstMutationEngineTests|VacuousMutationLegTests|
CertificationGateTeethTests|BrickAnalyzerReferenceTests) plus every touched class
(TrustLoopRecordSchemaTests, SessionExecutionBackendTests, SchemaVersionFloorTests,
RepairFeedbackTests, BoundedCandidateExecutionTests): 165 passed, 0 failed, net8.0.
Known limits, stated rather than hidden: a background thread that crashes the runner AFTER its
unit finished is attributed to whichever unit was executing when the process died; the
loader still instantiates the brick type in-process (its constructor runs in the certifier,
the witness legs no longer do); the timeout defaults are per-case 5s / per-unit 30s / total
10min and a witness whose honest cases need longer must raise them on the gate.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com># Conflicts: # docs/CertificationGate.md # samples/certified-brick-reuse/Ashlar.Certified.DamageResolver/certification-record.json # src/Ashlar.Infrastructure/Certification/BrickMutationEngine.cs
…body in scope # Conflicts: # samples/certified-brick-reuse/Ashlar.Certified.DamageResolver/certification-record.json # src/Ashlar.Infrastructure/Certification/AstMutationCatalog.cs
# Conflicts: # docs/CertificationGate.md
…geResolver record The gate, ledger, export and daemon lanes were merged in that order. Each merge kept both intents; this commit carries the fixes the merged whole needed that no single lane could have seen. AstMutationCatalog: the export lane's catalog now discards a mutant that does not compile instead of letting the engine credit the compiler's rejection to the witness. That exposed a latent defect in negate-condition: `.WithTriviaFrom(node.Condition)` was applied to the whole if-statement, so an `if` sitting under `#if NET8_0` lost the directive from its leading trivia and the mutant had a dangling `#endif`. Before, that mutant was silently "killed"; after export it vanished, and the ledger lane's The_catalog_mutates_the_branch_the_build_compiled (which needs a negate-condition on the guarded `if`) failed. The trivia now goes on the new condition expression, so the statement keeps its own. The mutant re-parse in DiscardNonCompiling also uses the candidate's own parse options — the build's symbols — so the same `#if` branches are code when the mutant is re-bound as when it was collected (the ledger lane's point). BrickMutationEngine and CollectMutations: the ledger's BrickCompileOptions is threaded through the gate lane's child-process structure (CollectMutations, CompileMutantAsync, FenceWouldRejectAsync, BindCandidate); the in-process mutant path the gate lane deleted stays deleted. BrickAnalyzerReferenceTests: the doc-shape test hard-coded `Version="0.1.1"` in the Ashlar.Analyzers snippet; the daemon lane deliberately shows `0.1.2` there because the 0.1.1 package ships no analyzers leg. The test's invariant is the reference SHAPE (ExcludeAssets="runtime;compile", never PrivateAssets="all"), so it now matches any version. Record: regenerated with the merged tool. 6 mutants, 6 killed, 0 survivors, no timeouts, no crashes; analyzerAssemblyVersion follows the VERSION bump to 0.1.2.0. Down from 10 because write-only constructor assignments are out of scope and non-compiling removals are no longer counted. HelloBrick certifies 2/2. ProjectB verifies the record and prints TRUSTED finalDamage=40. Docs: one schemaVersion bump (3, from the gate lane), one record shape; the mutation-leg row and CHANGELOG say a hung mutant is filed under timedOutMutants, not "scored killed", and the two Unreleased "Fixed" sections are one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he cert-gate
Every attack that ever beat the gate lived under /tmp in one dev container: the
fifthway twins whose byte-identical source certified under different DefineConstants,
the #if NET leak the fence never saw, the countdown loop whose mutant hung the certifier
forever, the recursive helper whose mutant killed it with a stack overflow, the toothless
witnesses a per-kind mutant cap could not expose, the helper bodies the catalog never
mutated. Each was fixed, and each fix is pinned by a unit test that rebuilds the attack
from a string. The attacks themselves were one `docker rm` from gone, and a fix without
its attack beside it is a claim the next refactor can silently unmake.
tests/adversarial-corpus/<id>/ now holds each attack as it was run — the brick source,
the consumer-shape project (net8.0, one PackageReference to Ashlar.Brick.Contracts,
ManagePackageVersionsCentrally=false, the shape of samples/hello-brick), the witness —
plus an expect.json naming the verdict the gate SHOULD give: ADMIT, REJECT with the leg,
REFUSE with a fragment of the refusal message, or VERDICT (either signed outcome) for the
two factorial bricks whose only remaining defect is the harness's own equivalent-mutant
false rejection, which knownIssue names. Expectations are robust to the catalog widening:
never a mutant id, a hash or a count, but which code a survivor sits in
(reasonContains "Resolve("), which branch it must not name (reasonNotContains
"baseDamage - armor"), which options the record must disclose (compileOptionsContains
"ASHLAR_EVIL", "NET8_0", "checkOverflow=true"), a floor on timed-out or crashed mutants,
a wall-clock ceiling on the gate. The directory sits at the repo root, outside every
project, behind an empty Directory.Build.props shield exactly like samples/, so no test
project's glob ever compiles a fixture; each is built by BrickCertificationProjectLoader
the way the gate builds a real brick.
AdversarialCorpusTests is a Theory over every directory there, driving LoadAsync then
CertificationGate as ShippedSampleCertificationTests does, and asserting expect, leg and
invariants. A directory without expect.json fails by name rather than being skipped
(confirmed with a temporary zz-no-expect directory: 22 rows, 21 passed, that one failed
with the designed message). expect.json is read with unmapped members disallowed, so a
misspelt invariant is a failure, not a silently vacuous expectation. Every verdict is also
held to the round-8 accounting: killed, timed-out, crashed and surviving mutants partition
the total. The class carries Tier=Build beside Category=Certification so a fast tier can
leave it out without leaving the cert-gate.
Twenty-one fixtures. Every one was run through tools/Ashlar.CertifyBrick built from this
tree first, and every current verdict is the one the gate should give, so nothing here
encodes a wrong answer. Two fresh fixtures come from the round-8 lane tests rather than a
/tmp oracle: exit-zero (Environment.Exit(0) in ExecuteAsync, which used to end the test
host with exit code 0 and no failure) and ifnet8-backdoor (the EY backdoor under #if
NET8_0 with a stock csproj). One REFUSE fixture, props-injection, is the
Directory.Build.props shape samples/ itself used to carry, with the payload in _shared/,
which the theory skips by its underscore. Three deviations from the oracles are recorded
in the README: the backdoor paths are temp-local so the leg reached does not depend on
/tmp/fifthway/out existing, the payroll pair carries the payroll2 witness (the one under
which both twins admitted), and every project is renamed Corpus.<Id> for a unique
assembly name.
The README documents the schema and the standard step this commit exists to establish: a
finding that survives adjudication becomes a fixture here in the same PR that fixes it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>…d the record Both child processes the gate spawns inherited the certifier's full environment minus a few names: EvaluatedBrickProject.Invoke removed eleven MSBuild property names before `dotnet msbuild -restore -t:Build` over the author's project, and LocalProcessExecutionBackend.RunChildAsync removed DOTNET_STARTUP_HOOKS before launching the witness replay runner. The record signing keys live in environment variables (ASHLAR_CERT_DEV_HMAC_KEY, ASHLAR_CERT_ED25519_KEY), so an <Exec> target in the brick's own .csproj, or the candidate's ExecuteAsync, could read the key that was about to sign its own certificate and mint one over any source text. Reproduced three ways at 73a704a: a fixture target's `env` dump carried the key value; a brick returned the key as an output from inside the runner; a candidate that forged an admitted record from inside the runner produced one that verified under the operator's key. ChildProcessEnvironment is the one helper both spawns now use. It clears the inherited environment and copies an explicit allowlist by name — PATH, HOME, temp, locale, the Windows system locations, the DOTNET_* host-location and first-run variables and the NUGET_* cache locations the SDK needs to restore and build, proxy settings, MSBUILDDISABLENODEREUSE — sized by making restore+build and the runner work under it, not by passing DOTNET_*/NUGET_* wholesale. Anything under ASHLAR_CERT_ is refused by prefix even if a future edit lists it. Deliberately absent: DOTNET_STARTUP_HOOKS, the MSBuild property names the old denylist chased, the SDK-resolver family that points a nested evaluation at the host's SDK, and DOTNET_ROLL_FORWARD — the runner's own runtimeconfig decides its runtime, and this repo's own container exports LatestMajor, which was rolling net8.0 children onto .NET 10. What this does not close, said in the code and in the docs: on Linux a same-uid child can still read /proc/<certifier pid>/environ, the certifier's initial block, which is where a key exported from the launching shell lives. Closing that means not holding the key in the environment at all (a key file readable only by the certifier's uid, or a signer the certifier calls and never sees the key of), children under another uid or in their own PID namespace, or hidepid=2. ChildProcessEnvironmentTests.KnownGap_… asserts the gap is open, so it reads as open and goes red when a platform closes it. The signed record now says who produced it, in strings that were already under the signature (no schema bump). Every execution pass carries `certifier=<assembly>/<informational version>;sourceRevision=<sha|unstamped>;certifierMvid=<mvid>`: the module version id is there because a dev build stamps the SDK default 1.0.0 and SourceLink is not wired, so under Deterministic builds the MVID is the only thing that identifies the exact gate binary. The mutation pass adds `mutationCatalog=<AstMutationCatalog.CatalogVersion>`; the constant starts at 2 (1 being the never-stamped 0.1.1 catalog) and AstMutationCatalogVersionTests pins the kind list and one fixture's exact mutant ids to it, so a kind or scope change without a bump fails and a bump without a re-pin fails too. A record re-labelled to another certifier or catalog version does not verify. A request with no build behind it (a hot-swap generation, a generated candidate) records the compile-options input as `default;reason=no-build` instead of omitting it: an absent input reads as "not recorded", and "compiled under the defaults because there was nothing to match" is a fact about the program judged. TrustLoopRecordSchemaTests' inputs count moves 2 -> 3. Docs: one section in docs/CertificationGate.md (since 0.1.2) on the child environment, the /proc limit and the certifier identity. samples/certified-brick-reuse/README.md already points at the record for the mutant count; nothing to change there. Tests, each red at 73a704a for the reason it names: TheAuthorsBuild_SeesNeitherTheSigningKeys…, TheReplayRunner_SeesNeitherTheSigningKey…, ACandidateThatForgesARecordFromInsideTheRunner…, AnAdmittedRecord_NamesTheCertifierAndTheMutationCatalog…, ARequestWithNoBuildBehindIt_SaysSoInTheCompileOptionsInput. A canary variable that is neither a secret nor allowlisted is asserted beside the keys in both spawn tests, because "the keys are gone" is what a denylist proves and "nothing else crossed" is what an allowlist proves. Full cert-gate: 553 passed, 0 failed (545 before, +8). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r-compile cache
Every round pays the cert-gate several times over. This commit makes the gate faster
without changing what it asserts, and puts a convention test on the one new seam that
could rot.
xunit.runner.json: maxParallelThreads 2 -> 4. The 2 dated from a pool-starvation flake
(FileCertificationRecordStoreConcurrencyTests flooding the pool beside
CertifiedBrickHotSwapHostTests' GC-bounded wait); that test owns its threads now, and
three consecutive full runs at 4 on the shared dev VM were 554/554 with no flake at
87 s, 95 s and 103 s wall (test phase 1.28, 1.39, 1.55 min) against 137 s (1.83 min) at
2 on the same tree. Two more runs at 2 with the cache in place came out at 132 s and
149 s, so the thread count is where the time goes.
Tiers. Six classes drive BrickCertificationProjectLoader.LoadAsync, which shells out to
a real `dotnet msbuild` (restore + build, tens of seconds each) where the rest of the
gate is Roslyn in memory: the two loader test classes, BrickCompiledSourceSetTests,
BrickEvaluatedCompileSetTests, CompiledProgramParityTests and
ShippedSampleCertificationTests. They carry [Trait("Category", "SlowTier")].
cert-gate-config.sh gains CERT_GATE_FAST_FILTER, composed as
"(${CERT_GATE_FILTER})&Category!=SlowTier" so a namespace added to the gate is in both
tiers, and cert_gate_list_tests honours CERT_GATE_COUNT_FILTER so the zero-test guard
sizes itself to the tier that ran. run-cert-gate.sh accepts --fast, writes
test-results/cert-gate-fast.trx so a fast run never overwrites a full one, and refuses
any other argument. CI is untouched and runs the full filter: 554 tests full, 490 fast.
SlowTierConventionTests keeps the split honest. It parses every test source the gate
selects with Roslyn and classifies by TOKENS, not text — a class is slow-tier when its
code uses BrickCertificationProjectLoader or EvaluatedBrickProject, or starts a process
and names a scripts/*.sh or a `dotnet` command line; a <see cref> in a doc comment or a
script path in an assertion message does not count, because reading a script is not
running it. Any such class without the trait fails the test by name. Two more facts pin
the filter line's shape in the script and that cert-gate.yml never passes --fast. At
73a704a the first fact fails naming all six classes and the second fails on the
missing filter.
Runner-compile cache. LocalProcessExecutionBackend compiled the witness replay runner
from the same source against the same reference set on every EnsureRunnerAsync.
RunnerCompileCache keys the compiled image by content — the runner source, the hosting
runtime, and every reference in compiler order by full path AND module version id, so
a rebuilt assembly at the same path is another key and an unkeyable in-memory reference
never hits — in a directory named by process id and start time that is removed on exit.
Misses on one key serialise on a per-key gate so a burst of identical backends compiles
once; the image lands in the cache by staging-then-move so no reader sees a half-written
file, and a hit whose file vanished under it falls back to a compile rather than a
failed certification. RunnerCompiledFresh on the backend and CompilesFor(key) on the
cache make the count observable; RunnerCompileCacheTests pin same-set reuse (two
backends, one compile, byte-identical images), recompile on a different set, on a
rebuilt assembly and on one changed character of source, and one compile for six
concurrent racers.
Measured, so the claim is the right size: a traced full run materialised the runner
137 times — 13 compiles (13 distinct reference sets; median 165 ms, max 798 ms) and 124
hits — so the cache removes about 20 s of CPU from a gate run, roughly 5 s of wall at
four threads, inside the noise of a VM shared by three lanes. A single CLI certification
gains nothing: the gate hands its backend to the mutation engine, so one certification
was already one compile (HelloBrick 3.9 s, DamageResolver 4.4 s, interleaved
before/after under equal load). The cache pays in any process that certifies
repeatedly — the test host, a daemon, an autonomy loop — not in `ashlar certify`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>…er-process runner-compile cache
…e named
Two lanes met on the integration branch with two names for the same idea.
r9/corpus tagged AdversarialCorpusTests [Trait("Tier", "Build")] so "a
fast-tier filter" could skip it; r9/speed actually built that filter --
CERT_GATE_FAST_FILTER subtracts Category!=SlowTier -- and added
SlowTierConventionTests, which parses every class the gate selects and fails
naming any that reaches BrickCertificationProjectLoader / EvaluatedBrickProject
or starts a dotnet process without [Trait("Category", "SlowTier")]. On the
merged tree that test would name two classes: AdversarialCorpusTests (its
Tier=Build was referenced by nothing) and r9/hygiene's
ChildProcessEnvironmentTests (EvaluatedBrickProject.Build plus real child
processes). Both now carry the one name the scripts and the convention test
agree on; Tier=Build goes, since a trait no filter reads is a promise nobody
keeps. The corpus README says the same.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>…parallel threads First full cert-gate on the merged tree (speed x hygiene x corpus): 582/583, CompositionCertificationGateTeethTests.CorrectComposition_WeakWitness_Rejects_WithStructuralTeeth failed on "constituents" where "mutation" was expected. Not a flake in the composition gate: CertificationRecordSigning.ResolveKey reads ASHLAR_CERT_DEV_HMAC_KEY from the process environment on every Sign and every Verify when the signer was built without an explicit key -- which is how CompositionProbeFixtures builds it. r9/hygiene's ChildProcessEnvironmentTests sets that variable process-wide for the whole life of a real child build or process (seconds), and was not in the serialized "EnvironmentVariables" collection that CertificationRecordSignerDevKeyTests already uses for the same variable. Under r9/speed's four xunit threads a composition test admitted its constituents under the dev key, then verified them while the operator key was set: "invalid certification signature", so the constituents leg rejected first. The class joins the collection (DisableParallelization = true: xunit 2.9 runs those after every parallel collection has finished, alone), so the variable it sets is seen by nobody else. The assertion that failed now carries the record's reason; the one that fired named no violation and cost a source reconstruction. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The tracked certification-record.json was exported before r9/hygiene landed. That lane put the certifier's identity into strings that were already under the signature — every execution pass now carries `certifier=<assembly>/<version>;sourceRevision=<sha>; certifierMvid=<mvid>`, and the mutation pass names its catalog (`mutationCatalog=2`) — so a record exported from the merged tree differs from the tracked one in more than timestamp and signature. The sample is the thing the README tells a reader to inspect; it should look like what the tool on this tree writes. Re-exported with tools/Ashlar.ExportCertifiedBrick built from 028f587 against the host checkout. Unchanged, as expected: contentHash, 6 mutants / 6 killed / 0 surviving, the witness and compile-options input hashes. Changed: the three execution-pass configuration strings above, the timestamp, and the signature over them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… container Add ashlar dogfood campaign and make every make dogfood-* target enter the repo's dev/test container so the .NET SDK is not a host install. A release manager dispatches docs-drift, regression, and developer-tool sub-agents; silence is fail-closed. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
The loader validates Interval before it can complain about a missing release manager, so the fixture must be a well-formed agent document. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
That page is the 0.1.1-vs-0.1.2 authority and its PackageReference snippets are checkout pins, not nuget.org install instructions. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
Specialist `dotnet test` left VBCSCompiler / MSBuild node-reuse processes running, and Environment.Exit then never returned from the CLI. Return ctx.ExitCode instead and disable node reuse for campaign children. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
Keep the automated dogfood campaign and take master's 0.1.2 published pin, product-split docs, and certification-gate compile authority. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
Master's closed-world static class and the in-tree build-options record share a name. Alias the record so Infrastructure compiles again. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
The campaign branch carried the pre-0.1.2 in-process compile-options work. Mixing it with master's closed-world certifier does not compile. Restore master's certification tree and drop the leftover files. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
Those cases called APIs that live only on the pre-merge branch. Master's DoctorCommand and SelfExtendCommand do not expose them. Co-authored-by: IanFrelinger <IanFrelinger@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an automated dogfood campaign so Ashlar can validate itself as a developer tool: a release manager dispatches specialist sub-agents (docs-drift, regression, developer-tool). Every specialist must report back; silence is fail-closed.
All
make dogfood-*work runs inside the repo's dev/test container. The .NET SDK lives in that image (mcr.microsoft.com/devcontainers/dotnet:10.0-nobleplus the ASP.NET Core 8 runtime). Do not install an SDK on the host.Changes
ashlar dogfood campaignplusmake dogfood-campaign/bash scripts/run-dogfood-campaign.shscripts/run-in-devcontainer.sh— idempotent wrapper overscripts/handoff/devbox.shmake dogfood-*target enters the container (nested make is a no-op once inside)docs/background-agents/examples/dogfood-campaign.json.cursor/agents/report to the release managerci/published-versionpins nuget.org so docs cannot sell unpublishedVERSIONapps/release-manager/docs updated to the extracted repo + in-tree campaign.github/workflows/dogfood-campaign.ymlruns the campaign in the same containerTesting
DogfoodCampaignTestsagainst this repositorybash scripts/run-dogfood-campaign.sh(Docker +ashlar-devtest:local)Testing strategy (blast radius)
dotnet test src/Ashlar.Tests.BackgroundAgents --filter FullyQualifiedName~Campaigninside the containerDogfoodCampaignTestsbash scripts/run-dogfood-campaign.sh