Skip to content

docs: rainix.yaml runs the four checks, and name the gates it adds - #120

Merged
thedavidmeister merged 2 commits into
mainfrom
2026-08-16-issue-88
Aug 17, 2026
Merged

docs: rainix.yaml runs the four checks, and name the gates it adds#120
thedavidmeister merged 2 commits into
mainfrom
2026-08-16-issue-88

Conversation

@thedavidmeister

@thedavidmeisterthedavidmeister commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

Closes#88

Docs-only, and confined to the two lines the issue names (README.md:55-56).

The claim that was false

The README listed forge test, forge fmt --check, slither . and
reuse lint as "Checks, each of which CI also runs", then said:

On top of the above, CI applies rainix's org-wide static checks via
.github/workflows/rainix.yaml.

rainix.yaml is not "on top of the above" — it is the only thing that runs
the above. This repo has three workflows; the other two are build-pointers
and package-release, neither of which runs a check. So a reader who took the
sentence at face value was left looking for a phantom source for the four, and
the gates that genuinely are additional went unnamed.

Falsified against rainlanguage/rainix@main

.github/workflows/rainix.yaml here is six lines and calls one reusable,
rainix-sol.yaml, which fans out to three:

rainix-sol.yaml:27 uses: ./.github/workflows/rainix-sol-static.yaml
rainix-sol.yaml:29 uses: ./.github/workflows/rainix-sol-legal.yaml
rainix-sol.yaml:31 uses: ./.github/workflows/rainix-sol-test.yaml

Between them those three run all four of the README's checks:

rainix-sol-static.yaml:51 ... sol-shell -c slither .
rainix-sol-static.yaml:52 ... sol-shell -c forge fmt --check
rainix-sol-legal.yaml:19 ... sol-shell -c reuse lint
rainix-sol-test.yaml:84 ... sol-shell -c forge test -vvv

The genuinely additional gates are the remaining six steps in
rainix-sol-static.yaml, and none of the four documented checks covers any of
them:

rainix-sol-static.yaml:19 actions/no-ignored-tests@main
rainix-sol-static.yaml:21 actions/no-submodules@main
rainix-sol-static.yaml:28 actions/agent-context-cap@main
rainix-sol-static.yaml:32 actions/frozen-snapshots-append-only@main
rainix-sol-static.yaml:36 actions/no-custom-natspec@main
rainix-sol-static.yaml:54 ... sol-shell -c rainix-sol-single-contract

The other steps in that job — nix-cachix-setup, the Foundry build cache and
forge soldeer install — are setup, not gates, so they are not listed.
rainix-sol-legal.yaml and rainix-sol-test.yaml add nothing beyond
reuse lint and forge test.

This PR's own check list confirms the fan-out at runtime, not just on paper —
rainix.yaml produces exactly three jobs, and the repo's only other checking
workflow is unrelated to the four:

build-pointers / copy-artifacts = SUCCESS
rainix / static / static = SUCCESS
rainix / legal / legal = SUCCESS
rainix / test / test = SUCCESS

Where the issue's proposed fix was wrong, and what I wrote instead

The proposed replacement described the six as gates "with no local equivalent".
Measured in this repo's own shell, that is literally true today:

$ nix develop -c bash -c 'for c in rainix-static rainix-sol-single-contract \
rainix-sol-artifacts slither reuse forge; do \
printf "%s -> %s\n" "$c" "$(command -v $c || echo ABSENT)"; done'
rainix-static -> ABSENT
rainix-sol-single-contract -> ABSENT
rainix-sol-artifacts -> /nix/store/2m5jljlm09jlagzslixbmrcw3k5v581q-rainix-sol-artifacts/bin/rainix-sol-artifacts
slither -> /nix/store/c8gqgnqzagbpaxx9f76m1q4d3lirvsq5-python3.13-slither-analyzer-0.11.5/bin/slither
reuse -> /nix/store/7vlrdv22ai5ajs2p798md5a13g3j7p8q-python3.13-reuse-6.2.0/bin/reuse
forge -> /nix/store/zb1ia37q7f8w8dg8ywxsk8mvw6fps7gd-foundry-0.0.0/bin/forge

But it is true for a reason that has nothing to do with the gates. flake.lock
pins rainix at f22d4dca, which predates both binaries. At the SHA CI itself
uses — RAINIX_SHA: 53e96a7d0a97d7c7c75c3b2412521324776fdac6, the pin inside
rainix-sol-static.yamlsol-shell carries both: at that SHA rainix's
flake.nix:304 puts rainix-sol-single-contract in sol-tasks, :319 puts
rainix-static in common-shell-inputs, and :622-623 builds sol-shell
from sol-build-inputs ++ sol-tasks ++ common-shell-inputs. So
nix develop -c rainix-sol-single-contract, and rainix-static for the
submodule, context-cap and snapshot gates (which are what those three composite
actions shell out to), become local equivalents the moment the lock is
refreshed.

Writing "no local equivalent" would have documented this repo's stale lockfile
as if it were a property of the org's gates: false the next time anyone runs
nix flake update, and enshrining the staleness in the meantime. The landed
sentence says "that none of the four covers" instead — which is the claim
that actually matters to a reader (a green local run of the documented checks
is not enough), and which stays true regardless of what the lock pins.

One smaller departure: the last gate is named by its rule, not its task
name.
The proposal said "a single-contract check"; the README says "one
contract per .sol file". rainix-sol-single-contract is a task name, which
is not something a reader can act on, and PR #51 already established here that
naming rainix task names in this README is how it rots.

The count is deliberately absent too — "six" would be a number in a README, the
exact shape of claim that #42 was filed to kill.

The change

-On top of the above, CI applies rainix's org-wide static checks via-[`.github/workflows/rainix.yaml`](.github/workflows/rainix.yaml).+[`.github/workflows/rainix.yaml`](.github/workflows/rainix.yaml) is what runs+all four in CI, via rainix's `rainix-sol.yaml`. It also applies org-wide gates+that none of the four covers — no ignored tests, no git submodules, an agent+context cap, append-only frozen snapshots, no custom NatSpec, and one contract+per `.sol` file — so a green local run is necessary but not sufficient.

Regions of README.md deliberately left alone

Sibling agents hold adjacent findings in this same file, so the diff is exactly
lines 55-56 and nothing else. Untouched:

The new paragraph also stays in place at 55-56 rather than moving up to sit
directly under the check list. Reordering would grow the diff across a region a
sibling may be editing, for a readability gain the issue did not ask for.

QA

  • Discriminating tests: n/a, and not skipped for convenience — the diff is
    five lines of English prose in a README, adding no branch, selector or code
    path a forge test case can assert over. The only test shape that could bind
    it would fetch rainlanguage/rainix@main over the network from inside a
    Solidity test via ffi and diff its workflow steps against the README's word
    list — a docs-lint harness, which per the org's "rainix owns shared CI" rule
    belongs in rainix as a reusable, not hand-rolled here. What stands in for the
    red-then-green run: the old sentence's assertion ("on top of the above") was
    executed against the workflow graph and is false — rainix.yaml
    rainix-sol.yaml{static, legal, test} runs all four checks itself, at
    the line numbers quoted above — and every assertion in the replacement was
    then executed the same way (fan-out, the four check lines, the six gate
    steps, and that no other step in any of the three reusables is a gate).
  • Mutations applied: n/a as line-breaking, run as claim-breaking — there is
    no line here whose inversion a test could kill. Each of the six named gates
    was negated in turn and the negation checked against rainix-sol-static.yaml
    (all six survive as real steps, at :19, :21, :28, :32, :36, :54);
    the "none of the four covers" clause was attacked by hunting for a local
    command that does cover them, which is what surfaced the flake.lock /
    RAINIX_SHA divergence and moved the wording off the issue's "no local
    equivalent". Full suite on this branch, in the repo's own pinned shell:
    forge test — 16 suites, 134 tests passed, 0 failed, 0 skipped;
    forge fmt --check — exit 0; reuse lint — 47/47 files, compliant with
    REUSE 3.3; slither . — 9 contracts, 98 detectors, 0 results.
  • Category check: the issue names one wrong sentence (README.md:55-56) and
    cites :42-48 as the list it misdescribes. The category is "a statement in
    this README about what CI runs that the workflow graph contradicts", so all
    three of this repo's workflows and all three rainix reusables they reach were
    swept, not just the named lines. One wrong statement found and fixed; :42-48
    re-verified as correct and left as-is (all four commands exist in the shell
    and all four are run by CI); build-pointers.yaml and package-release.yaml
    confirmed to run no check, which is what makes "on top of the above" have no
    possible referent. Every gate step in the three reusables is named in the new
    sentence — none was dropped for brevity.
  • Oracle: rainlanguage/rainix@main's workflow files and flake.nix at
    53e96a7d, read directly, plus this repo's .github/workflows/ and
    flake.lock. Not the issue body's restatement of them: the issue said the
    reviewer named four of six gates and two were confirmed on re-measurement,
    and all six were re-measured here independently.

Found, not touched

flake.lock pins rainix f22d4dca; rainix-sol-static.yaml runs its checks in
rainix 53e96a7d. The local shell is therefore not the shell CI uses —
rainix-static and rainix-sol-single-contract exist in the latter and not the
former. That is outside issue #88 and is not filed or fixed here.

The README said "on top of the above, CI applies rainix's org-wide static
checks via .github/workflows/rainix.yaml". rainix.yaml is what runs the four
checks listed above it, not a layer on top of them, and the gates that are
genuinely additional went unnamed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@thedavidmeisterthedavidmeister self-assigned this Aug 16, 2026
@coderabbitai

coderabbitaiBot commented Aug 16, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@thedavidmeister, you've reached your PR review limit, so we couldn't start this review.

Next review available in:57 minutes

Limit details: You’ve used all 1 included review currently available under your plan.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: d5b89d2c-ac2b-4820-ace5-b180d6ff33df

📥 Commits

Reviewing files that changed from the base of the PR and between 308ad50 and 527cfe1.

📒 Files selected for processing (1)
  • README.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@thedavidmeister

Copy link
Copy Markdown
ContributorAuthor

@coderabbitai review

@coderabbitai

coderabbitaiBot commented Aug 16, 2026

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@rainlanguagerainlanguage deleted a comment from coderabbitaiBotAug 16, 2026
@thedavidmeister
thedavidmeister merged commit 03a64fe into mainAug 17, 2026
5 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

README's "On top of the above" mis-attributes the four listed checks away from rainix.yaml, which is what runs them

2 participants

@thedavidmeister@claude