Uh oh!
There was an error while loading. Please reload this page.
feat(pm): record a passive boot marker on every verify-lock invocation, so cross-restart evidence about /tmp accumulates by itself - #13145
Open
os-elon wants to merge 4 commits into
Open
Conversation
os-elon
marked this pull request as ready for review
August 29, 2026 06:42
os-elon
enabled auto-merge
August 29, 2026 06:42
github-merge-queueBot
removed this pull request from the merge queue due to failed status checks
Aug 29, 2026
Uh oh!
There was an error while loading. Please reload this page.
Contributor
⛔ merge queue 构建失败 — 先分诊,再决定要不要重排队列构建 33241594487 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集), 失败的 job(日志抽取,best effort):
跨 PR 相同签名(24h,按失败测试文件聚合):
历史信号:
分诊清单:
Generated by Claude Code · merge-queue-triage workflow (#4859) |
This was referenced Aug 29, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #12919. It builds the mechanism the ruling ordered and deliberately writes none of the card's conclusions, so the card stays open to receive readings — that is the half left behind, and it is left behind on purpose.
The ruling, and the one word in it that points the wrong way
The maintainer ruling (2026-08-29) says "adopting option B (piggyback passive measurement)", but triage's own table labels A as the passive marker and B as the active multi-container fleet campaign. The ruling's description is unambiguous in the other direction — "piggyback passive measurement", "no dedicated multi-container restart campaign", "the measurement happens because of what gets typed, not because someone remembers to run it". This PR implements the description: the passive marker. No fleet campaign, which would need explicit authorisation in any case.
First, the thing triage said had to be measured before the design
Triage's confidence gap 3: if the ledger rolls faster than the restart interval, a passive marker is truncated before it is ever read. Measured on this container before anything was written:
LEDGER_MAX_BYTESreadonlyconstant — not env-overridableSo the bound sits roughly three orders of magnitude away from the restart interval. The hazard triage flagged does not bite at the observed rate.
ledger_appendreturns early when the file exceeds the bound — it does not trim, roll or rotate. Nothing is truncated; new records simply stop being written. So the failure mode a marker in the ledger would have had is not "truncated before it is read", it is "never written after the bound" — silently, at exactly the point the population became long enough to be worth reading. Same remedy, different reason, and the reason is now in the code.The mechanism
A second file beside the lock,
os-heavy-verify.lock.boots, plus a fixed-name probe.boot_marker_observeruns inmainon every invocation — a run, a refusal, a--status, even a usage error — before dispatch. It appends one record per distinct boot, never one per run.Where it lives is the measurement. The marker sits in the same directory as the ledger it is about, so it is itself subject to whatever happens to that directory across a reset. A marker written somewhere known-durable would measure nothing. It satisfies all three: no size bound of this file's can stop it being written, normal operation writes it without anyone remembering, and it is exposed to the phenomenon under study.
The boot identity is the kernel's, not a clock subtraction.
/proc/sys/kernel/random/boot_idis exact, needs no tolerance, and jitters not at all — the derived boot is still recorded, as the quantity the two existing readings used, but it is no longer the key. Where no boot id exists the key falls back to the derived boot with a tolerance and the field records-, so a reader knows which key was used.Both controls travel on every record, because a reading without its controls is not a reading:
pos=: a file written at that instant, read back through the samestatand compared against the same derived boot, must readafter. Anything else means the comparison was not live for that reading and itsmarkerpre/ledgerpreare worth nothing.btime=andpid1after=: the kernel's own record of when it booted, and how long after the derived boot PID 1 started. A smallpid1aftersays the process tree restarted with the clock, so a file older than the boot means the filesystem did not restart with them. It can read slightly negative from whole-second rounding; what the control says is that the number is small in either direction.The strongest field is
prev=, and it carries no timestamp at all. It names the boot identity on the previous line of the same file. If it names a different boot, this file was present before that restart and is being appended to after it — evidence by content, with no clock comparison anywhere in it.markerbirth/markerpreandledgerbirth/ledgerprecorroborate it from the timestamp side and reproduce the exact row the two existing readings were filed over.Cost, measured and decomposed
It runs on every invocation in every agent container, so #12783's fence binds: nothing that outlives a process and needs reaping, and nothing that can redden a gate.
A/B against the same file without the mechanism, 100 invocations per arm, three interleaved rounds, worst case (
--show-budget, which does nothing else):The dominant term is bash parsing 691 added lines, most of them comment — a cost this file pays for any comment. Against a run that holds this lock for tens to hundreds of seconds that is under one part in a hundred thousand. Shared-box milliseconds, per this file's own doctrine.
The fast path forks nothing on bash 5.0+:
/proc/uptimeand the boot id are read with thereadbuiltin, the marker with one builtin read. Counted rather than claimed — comments stripped and arithmetic$((excluded, the four fast-path functions contain exactly one command substitution, thenow_sfallback reached only below bash 5.0; the control, the same filter over the once-per-boot path, counts 12.Nothing here needs reaping. Two fixed paths, exactly as
.holderand.ledgeralready are: the marker is append-only and bounded at 65536 bytes, the probe is overwritten in place and never grows. Neither accumulates a file per process, which is what the fence was against. Every write is best-effort and every failure swallowed; a self-test case pins that an unwritable marker loses boots, never runs, and a second pins that it stays quiet about it.Concurrency
Several agents in one container invoke this concurrently. Appends are a single short
printfto an O_APPEND fd, well under PIPE_BUF, atomic on Linux — the same argument the ledger's format already rests on. The read-then-append sequence is not atomic and deliberately takes no lock: a lock round-trip here would cost more than the measurement. Two runs that both notice the same new boot can each append a record for it; that is benign, stated rather than prevented, and--reportgroups by boot identity so a duplicated boot reads as one boot — and a duplicate carries its own controls anyway.--self-testcase "and a kill -9 mid-run leaves nothing behind to reap either" asserted that nothing at all sat beside the lock path after an unlocked run. It went red, correctly, the first time the marker existed. That case's own comment states the invariant it protects: a hand-rolled lockfile is the option the 2026-08-22 ruling rejected, and a killed holder leaving the lock held is its whole cost. The blanket glob was a convenient spelling of that back when the only files that could be beside the lock were lock state.The marker is not lock state: nothing reads it to decide whether the lock is held, a
kill -9leaves it valid because an append either landed whole or did not land, and reaping it would destroy the measurement rather than tidy anything. So the assertion is re-spelled to the property it was protecting — no lock state survives an unlocked run — and paired with a positive that counts the residue and requires it to be exactly the marker pair, because a narrowed assertion that counts only what it chose to look at is how a pin stops catching the regression it exists for. Therun_unlockedheader now says the claim is about that function, not about the invocation.What it can and cannot conclude — printed with the numbers, not filed beside them
--reportgained a boot section that ends in a limits block, and--statusnames the marker so a reader meets it:prev=-. The negative direction is visible only off-box, comparing a reading carried away earlier against a later one from the samemachine=. That is whymachine=is recorded at all.machine=values, and the line phrased as that bound, never as "/tmp survives". One observed non-survival settles the other direction at k=1.⛔ What was deliberately not touched
The "the reason it sits there is left unasserted" wording in the
--reportscope block, and the "AND THE OPPOSITE PREMISE IS NOT CLAIMED" paragraph in the slots block. Two same-direction observations disqualify a mechanism; they do not establish its negation, and this PR adds a third same-direction observation without touching either sentence. Proven byte-for-byte rather than asserted — both blocks extracted from the base commit and from this branch and hashed:Nor was
platform-readings.mdtouched: that line is the future step, and this round builds the mechanism rather than writing a conclusion. Nor was the scope of #12795 entered — whether CPU-heavy gate runs should be routed through this lock is that card's question and is untouched here.The first datapoint
Taken from this container the moment the mechanism existed, in the format of the two readings already on the card. It is a third same-direction observation of the ledger outliving a reported boot — recorded automatically this time, with both controls, and carrying no conclusion.
markerbirth/markerpreread-on a container's first record, and correctly: the marker is created by that very append, so at that moment there was no marker to date. It says the same thingprev=-says. The full table is on the card.Verification
Union re-run after the final commit, at
05e7d98bfe. Exit codes captured before any pipe; each gate's own verdict line quoted rather than a bare status.bash scripts/pm/os-verify-lock.sh --self-test✓ os-verify-lock self-test: all cases pass.— 225 cases, up from 195 on the base commit, measured by running the base tree's own suite in a comparison worktree rather than by recalling a number. 33 cases added in source; 2 of them belong to the no-/proc/uptimebranch this host does not take, and one replaced case nets +1pnpm check:bash32-floor✓ check-bash32-floor: 22 tracked shell file(s) ... name no bash 4+ constructpnpm check:entry-guard✓ check:entry-guard: 177 scripts/ file(s)pnpm check:parse-guard✓ check:parse-guard: 176 scripts/ file(s)pnpm check:pnpm-filter-targets✓ check:pnpm-filter-targets: 140/177 --filter occurrence(s)pnpm check:agent-test-spelling✓ check-agent-test-spelling: 0 violationspnpm check:cli-command-ids✓ check-cli-command-ids: 294 command-id literal(s)pnpm check:watch-hint-literal✓ check-watch-hint-literal: 18 ROOT_DIR_WATCH_HINTS declaration(s)pnpm check:cross-package-test-inputsOK: 23 package(s) read outside themselvespnpm check:nul-bytescheck-nul-bytes: OK (scanned 7288 text file(s) ... no raw ASCII control bytes)node scripts/check-ci-filter-parity.mjsOK: all 121 declared cross-package glob(s)node scripts/check-shard-attestation.mjs✓ check-shard-attestation: 2 aggregate gate(s)Gate family derived with
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstackafter mergingorigin/main, so the list is not stale: 12 families matched. The twelfth,check-test-completeness.mjs, answers a standalone run with exit 1 and a usage line — an invocation error, filed as #13110, not a gate result and not touched here.pnpm lintwas narrowed, and the narrowing is measured rather than assumed. ① The population comes from eslint's own config, not from a guess: run against the changed path it answers "File ignored because no matching configuration was supplied". ② The count comes from--format json: 1 result, 0 errors, and the sole message is that ignore notice; the control, a real.mjspath, returns 1 result with 0 errors and no messages, so the instrument distinguishes "linted clean" from "not in the population". ③ Invariance for untouched files:eslint.config.mjscarries noparserOptions.projectand no typed rules, so a diff touching no JS or TS file cannot move any untouched file's verdict. CI runs the full sweep regardless.No changeset, derived rather than asserted.
pnpm check:published-filesis green over 69 publishable packages, and a sweep of all 75 workspace manifests finds zero whosefileswhitelist mentionsscripts— soscripts/pm/os-verify-lock.shreaches no tarball and nothing here is user-visible. The PR carriesskip-changeset.Generated by Claude Code
Generated by Claude Code