Uh oh!
There was an error while loading. Please reload this page.
Replace the inlined implementation copy in the identifier fuzz oracle - #106
Conversation
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Warning Review limit reached
Next review available in:34 minutes Limit details: You’ve used all 1 included review currently available under your plan. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
# Conflicts: # test/lib/LibCodeGen.requireContractName.t.sol
Uh oh!
There was an error while loading. Please reload this page.
Closes#60
testRequireContractNameAcceptedNamesAreIdentifiersrestatedLibCodeGen.requireContractName's own range comparisons character forcharacter. A copy of the implementation is not an oracle: it agrees with the
implementation by construction, so it cannot see a design error that is present
in both. The file already carries an independent oracle in
LibCodeGenSlow.isContractNameSlow, which decides by membership of an alphabetspelled out character by character. This test now uses it.
Path note: PR #56 is still open, so on
maintoday the file istest/lib/LibCodeGen.requireContractName.t.sol, and that is where this isedited.
The defect is silence, not redness
This finding cannot be reproduced as a test that goes red — the defect is that
the test stays green when it should be red. The discriminating experiment is
the paired edit a copy invites: move a boundary in the library, and reflexively
move the copy in the test to match. Both experiments below were run against
main's test body, before the fix.Paired mutation 1 — library and inlined copy both drop the
&& i > 0guard,so a leading digit is accepted:
The independent oracle catches it with the counterexample
"9". The copy doesnot, over all 2048 runs.
Paired mutation 2 — library
char <= 0x5Abecomes0x5Band the copy'schar <= "Z"becomeschar <= "[", so[is accepted:After the fix there is no copy left to move, so the same library-only mutations
are exactly M2 and M4 in the matrix below — both killed.
Mutation matrix
Every entry mutates one line, runs only
testRequireContractNameAcceptedNamesAreIdentifiers, then restores fromHEAD.[fuzz] runs = 2048.""LibCodeGendrops&& i > 0(leading digit allowed)"9"LibCodeGendrops the empty-name revert""LibCodeGenletter range0x5A→0x5B"["LibCodeGendrops$from the accepted set"ABC…xyz_$0123456789"M2 and M4 are the two design errors the pre-fix test was blind to above.
Proof the suite actually ran, rather than a filter matching nothing: the harness
asserts a
[PASS]/[FAIL]line for the target test name appears inforge'soutput, and reports
HARNESS-ERROR-OR-ZERO-MATCHotherwise. It never did — everyrow above carries a real counterexample and a run count. The first version of
that check demanded exactly one matching line, but
forgeprints each[FAIL]line twice (suite body and summary), so it mislabelled all five kills; the check
was corrected and the entire matrix re-run from scratch, with identical verdicts
and counterexamples both times.
Why this test is kept rather than deleted
Once the oracle is shared, this test's body is identical to
testRequireContractNameMatchesAlphabet; only the fuzzed type differs(
stringvsbytes). That difference is load-bearing, so I measured it ratherthan asserting it — instrumented copies of both generators, 2048 runs each,
counting names the reference alphabet accepts:
stringbytesThe
stringgenerator reaches the accepted half of the domain several timesmore often, and is the only one of the two that produced accepted names of
length 2 and 3. (Lengths 54 and 64 are the two alphabet constants themselves,
fed back in from the fuzz dictionary.) The docstring now states this as the
reason the test exists alongside its
bytestwin.Where the issue's proposed fix was adjusted
if (LibCodeGenSlow.isContractNameSlow(name)) {. I used the file's existingassertAccepted/assertRejectedhelpers for the two branches instead ofre-spelling the
vm.expectRevert(abi.encodeWithSelector(...))longhand, whichis what the neighbouring
testRequireContractNameMatchesAlphabetalready does.describe what the test now does and why its generator is a
string.bytesdomain is what makes the two tests differ. Measured, the
bytesdomain doesreach accepts too (11-13/2048): the difference is density and length, not
presence. Stated that way in the docstring rather than overclaiming.
QA
testRequireContractNameAcceptedNamesAreIdentifiers- itdoes NOT fail on base, and that is the finding: on base its oracle is a copy of
the implementation, so it agrees by construction. Verified by running two paired
mutations (library boundary moved AND the test's copy moved to match) against
base: the test PASSED all 2048 runs both times, while
testRequireContractNameMatchesAlphabet(counterexample0x39),testRequireContractNameEveryLeadingByte,testRequireContractNameEveryTrailingByte,testRequireContractNameRangeBoundaries,testRequireContractNameRejectsLeadingDigitand
testRequireContractNameRejectsOneBadByteFAILED. With the fix applied, thesame mutations kill it (M2, M4 below). Full suite on the branch head:
nix develop -c forge test= 134 passed, 0 failed, 16 suites.test/lib/LibCodeGen.requireContractName.t.sol:138if (LibCodeGenSlow.isContractNameSlow(name))-> condition inverted -> killed bytestRequireContractNameAcceptedNamesAreIdentifiers, counterexample"", run 0.(2)
src/lib/LibCodeGen.sol:51(isDigit && i > 0)->isDigit-> killed by thesame test, counterexample
"9", run 49. (3)src/lib/LibCodeGen.sol:43-45theempty-name revert -> deleted -> killed, counterexample
"", run 34.(4)
src/lib/LibCodeGen.sol:48char <= 0x5A->char <= 0x5B-> killed,counterexample
"[", run 854. (5)src/lib/LibCodeGen.sol:50char == 0x5F || char == 0x24->char == 0x5F-> killed, counterexample"ABC...xyz_$0123456789", run 111. 5 applied, 5 killed, 0 survived; each runrestored from
HEADand the tree verified clean afterwards. The harness assertsa
[PASS]/[FAIL]line for the target test name is present inforgeoutput,so a zero-match filter or compile error cannot read as "survived"; the whole
matrix was run twice end to end with identical verdicts and counterexamples.
LibCodeGenSlow.isContractNameSlow, which decides by membership ofSLOW_HEAD_ALPHABET/SLOW_TAIL_ALPHABET— the identifier alphabet spelled outcharacter by character in the test tree, never importing or restating the
char >= 0x41 && char <= 0x5A-style range arithmeticLibCodeGendecides with.Before this PR the oracle was a character-for-character copy of that arithmetic.
copy at lines 134-145 with the independent
isContractNameSloworacle whilekeeping the
stringgenerator; covered exactly that. The issue's incidentalobservation about the thin accept branch (101/2048) was measured rather than
acted on: 56 and 65 accepts per 2048 for
stringvs 13 and 11 forbytes,recorded in the docstring as the reason the generator stays a
string. Noother file, test or behaviour touched.
nix develop -c forge fmt --check— exit 0, no diff.