Skip to content

chore(audit): record the adversarial mutation test scan - #23

Merged
thedavidmeister merged 2 commits into
mainfrom
chore/mutation-test-scan-record
Aug 21, 2026
Merged

chore(audit): record the adversarial mutation test scan#23
thedavidmeister merged 2 commits into
mainfrom
chore/mutation-test-scan-record

Conversation

@thedavidmeister

@thedavidmeisterthedavidmeister commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Closes an adversarial mutation test run over the whole of this repo by recording
it where the org health check looks. This repo had audit/protofire/ but no
audit/mutation-test-scans.json, so the file is created here with the first
entry.

The run scanned a4692fddbb95a0018cec42969ef6c7fa1b0cf86c: 23 behaviours across
src/LibCast.sol and src/LibConvert.sol, 38 mutants authored in all, and 3
findings filed as #20, #21 and #22. The pre-existing suite was probed against the
36 that existed at that point — 27 killed, 9 survived — because M37 and M38 were
written later; that is why killedBefore + survivedBefore totals 36 rather
than mutantsProbed.

testsAfterCommit is set equal to commit, which is the documented encoding
for a run that landed nothing. This run was explicitly instructed not to merge
anything, so no coverage has reached the default branch. The 9 tests it added —
along with 4 existing ones strengthened in place — are in #18 and #19, both open.
The record says so in its own note field and states what should supersede it
once those land: an entry whose testsAfterCommit names the merged tree and
whose testsAfter reads 15 (LibCast 4 -> 9, LibConvert 2 -> 6). The alternative,
quoting an after-count that holds at no tree in the range, is exactly the
unfalsifiable record the template warns against.

Re-probing both PRs' tests together against all 38, on a green 15-test baseline,
is 37/38 killed with 0 no-run and 0 harness errors — the lone survivor being the
equivalent mutant M23 the record documents.

REUSE.toml lists audit/**/, which matches files inside subdirectories such as
the existing audit/protofire/…pdf but not a file sitting directly in audit/,
so the new path is listed explicitly.

QA

  • Discriminating tests: n/a - this PR adds a JSON record and one REUSE path, and changes no executable code, so there is no behaviour for a test to discriminate. The full suite was run on this branch and is green at 6 passed / 0 failed, unchanged from the base, confirming the diff is inert. jq -e . confirms the JSON parses.

  • Mutations applied: n/a - no executable code in the diff to mutate. The mutation results this record reports were produced against src/LibCast.sol and src/LibConvert.sol and are evidenced in test(LibCast): observe the intermediate and pin the in-place contract #18 and test(LibConvert): pin aliasing, source lifetime and the write bounds #19, which carry the per-mutant tables.

  • Oracle: the scan record template and field semantics in the adversarial-mutation-test README at skill version 0.34.0, which specifies the five must-have fields, the two-tree rule that every before number holds at commit and every after number at testsAfterCommit, and the rule that a run which landed nothing sets testsAfterCommit equal to commit rather than null or absent. publishedTag and commitsAheadOfTag were read from the repo with git describe --tags --abbrev=0 and git rev-list --count v0.1.0..main, not assumed.

  • Category check: no issue drives this PR; it is the closing step the skill requires of every run, clean or not. Covered: the timestamp, scanned commit, tests-after commit, published tag and distance, scope, tool and skill version, and a summary naming the filed issues. Not covered, deliberately: a merged-tree testsAfterCommit, because merging was out of scope for this run.

🤖 Generated with Claude Code

Adds the scan record the org health check reads to answer which release was
last audited. This is the first entry for this repo, so the file is created
here.
The record holds at a4692fd. testsAfterCommit is set equal to commit because
this run was instructed not to merge, so no coverage landed on the default
branch; the run's tests are in the two open PRs the entry names, and the entry
says what should supersede it once they land.
REUSE.toml covers audit/**/ which matches files inside subdirectories, so the
new file at the top of audit/ is listed explicitly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitaiBot commented Aug 21, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@thedavidmeister, you've reached your PR review limit, so we couldn't start this review.

Next review available in:42 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 92b6395e-d0f2-4ab4-a74c-e0c1df9009d1

📥 Commits

Reviewing files that changed from the base of the PR and between a4692fd and cc5a580.

📒 Files selected for processing (2)
  • REUSE.toml
  • audit/mutation-test-scans.json

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…t total
`testsAfter` was projected as 21 in the note. The two coverage PRs take
LibCast 4 -> 9 and LibConvert 2 -> 6, so the merged tree runs 15 — confirmed
by a combined probe whose baseline is green at 15 passed.
The note also called 15 "the run's 15 tests". 15 is the post-merge total; the
run added 9 and strengthened 4 existing ones in place.
`mutantsProbed` 37 -> 38 (M38 was authored late), with the note now explaining
why killedBefore + survivedBefore total 36 instead: M37 and M38 did not exist
when the pre-existing suite was probed. The combined re-probe over all 38 is
37/38 killed, 0 no-run, 0 harness errors, the lone survivor being the already
documented equivalent mutant M23.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@thedavidmeister
thedavidmeister merged commit 7676c38 into mainAug 21, 2026
4 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@thedavidmeister