Skip to content

docs(agents): add external skill precedence and evidence rules - #1306

Merged
BigSimmo merged 4 commits into
mainfrom
claude/frontend-checklist-skills-ece5e6
Jul 28, 2026
Merged

docs(agents): add external skill precedence and evidence rules#1306
BigSimmo merged 4 commits into
mainfrom
claude/frontend-checklist-skills-ece5e6

Conversation

@BigSimmo

@BigSimmoBigSimmo commented Jul 28, 2026

Copy link
Copy Markdown
Owner

The Front-End Checklist skill corpus (~390 skills) and the caveman output-style plugin are installed user-global and know nothing about this repo's contracts. Adds a tie-breaker section so generic guidance never overrides committed docs or tests.

  • Repo docs win on conflict: wiring-conventions, search-chrome-behaviour, rag-behaviour, the @theme tokens in globals.css, and any committed test.
  • Records the touch-target collision: generic WCAG 44px guidance maps to min-h-11, which hit a sub-pixel rounding flake in ui-smoke, so production tap targets use min-h-12. Design-scratch mockups still carry min-h-11 and remain gate-exempt.
  • Output-style compression never applies to verification evidence or calibration: paste the decisive gate line, state verified vs assumed, and treat third-party fix claims as unverified until checked.

Summary

Verification

  • npm run verify:pr-local

During development, use npm run verify:cheap as the faster iteration gate before the final PR-local preflight.

  • npm run verify:ui when UI, routing, styling, browser behavior, reduced-motion, or forced-colors behavior changed
  • npm run verify:release before release or handoff confidence claims

For retrieval, ranking, selection, chunking, source/citation rendering, or answer-contract changes, verify:pr-local runs eval:rag:offline automatically. Run the offline command directly during iteration before spending a live eval.

  • npm run eval:retrieval:quality (must stay 36/36) when retrieval, ranking, selection, chunking, or scoring behavior changed — CI cannot run it (needs live keys), so run it locally and paste the summary. A metadata/governance-weighting change once buried correct docs (recall 1.0→0.76) and only this eval caught it.
  • npm run eval:rag -- --limit 15 + npm run eval:quality -- --rag-only when answer generation, the synthesis prompt, or answer post-processing changed (grounded-supported must not drop; citation-failure 0)
  • npm run check:production-readiness when clinical workflow, privacy, environment, Supabase, source governance, or deployment behavior changed
  • npm run check:deployment-readiness when deployment startup, hosting, or rollout behavior changed

Risk and rollout

Complete this section for clinical, data, API, auth/privacy, workflow, dependency, build, or deployment changes.

  • Risk:
  • Rollback:
  • Provider or production effects: None / describe the explicitly authorized effect

Clinical Governance Preflight

Complete this section when the change touches ingestion, answer generation, search/ranking, source rendering, document access, privacy, production env, or clinical output.

  • Source-backed claims still require linked source verification before clinical use
  • No patient-identifiable document workflow was introduced or expanded without explicit governance approval
  • Supabase target remains Clinical KB Database (sjrfecxgysukkwxsowpy)
  • Service-role keys and private document access remain server-only
  • Demo/synthetic content remains clearly separated from real clinical sources
  • Source metadata, review status, and outdated/unknown-source behavior remain conservative
  • Deployment classification/TGA SaMD impact was checked when clinical decision-support behavior changed

Notes

Summary by CodeRabbit

  • Documentation
    • Added guidance on prioritizing repository documentation and tests over external instructions.
    • Clarified verification standards, including distinguishing confirmed results from assumptions and requiring concrete test evidence.

The Front-End Checklist skill corpus (~390 skills) and the caveman
output-style plugin are installed user-global and know nothing about this
repo's contracts. Adds a tie-breaker section so generic guidance never
overrides committed docs or tests.
- Repo docs win on conflict: wiring-conventions, search-chrome-behaviour,
rag-behaviour, the @theme tokens in globals.css, and any committed test.
- Records the touch-target collision: generic WCAG 44px guidance maps to
min-h-11, which hit a sub-pixel rounding flake in ui-smoke, so production
tap targets use min-h-12. Design-scratch mockups still carry min-h-11 and
remain gate-exempt.
- Output-style compression never applies to verification evidence or
calibration: paste the decisive gate line, state verified vs assumed, and
treat third-party fix claims as unverified until checked.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitaiBot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

AGENTS.md adds guidance for prioritizing repository documentation and tests over external instructions, and requires explicit evidence when reporting verification results or third-party fix claims.

Changes

Agent guidance updates

Layer / File(s)Summary
Precedence and verification guidance
AGENTS.md
Documents external skill precedence, requires decisive verification output, distinguishes verified from assumed statements, and treats third-party fix claims as unverified until checked.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

  • BigSimmo/Database#898 — Updates related AGENTS.md guidance for repository productivity skills and their application.

Suggested labels:codex

Suggested reviewers:cursoragent

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check nameStatusExplanation
Title check✅ PassedThe title clearly summarizes the docs-only change about skill precedence and evidence rules.
Description check✅ PassedThe description covers the change, includes the required template sections, and provides the key implementation details.
Docstring Coverage✅ PassedNo functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check✅ PassedCheck skipped because no linked issues were found for this pull request.
Out of Scope Changes check✅ PassedCheck skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/frontend-checklist-skills-ece5e6

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@supabase

supabaseBot commented Jul 28, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project sjrfecxgysukkwxsowpy because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@BigSimmo
BigSimmo enabled auto-merge (squash) July 28, 2026 02:51

@coderabbitaicoderabbitaiBot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
AGENTS.md (1)

229-236: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Scope “always win” to external guidance.

Clarify that repository contracts override generic skills/plugins only within this context and remain subordinate to higher-priority system, developer, user, security, and compliance requirements.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@AGENTS.md` around lines 229 - 236, Update the “Repo contracts outrank generic
rules” guidance in AGENTS.md to limit its precedence to external or generic
skills and output-style plugins. Explicitly state that repository contracts
remain subordinate to higher-priority system, developer, user, security, and
compliance requirements.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@AGENTS.md`:
- Around line 237-241: Update the guidance in the “Never regress a fixed flake…”
section to accurately distinguish WCAG 2.2’s 24×24 AA minimum from the 44×44 AAA
enhanced criterion. Describe min-h-12 as the repository’s tested, stronger
production baseline, while preserving the existing ui-smoke exception and mockup
exemption.
- Around line 256-257: Update the “Third-party fix claims stay unverified until
checked” guidance to prioritize inspecting available local refs and make git
fetch conditional on explicit user authorization. State that network access,
including GitHub/GitLab or provider-backed workflows, must not be used without
confirmation, while preserving the requirement to verify the ref before treating
claims as fact.
---
Nitpick comments:
In `@AGENTS.md`:
- Around line 229-236: Update the “Repo contracts outrank generic rules”
guidance in AGENTS.md to limit its precedence to external or generic skills and
output-style plugins. Explicitly state that repository contracts remain
subordinate to higher-priority system, developer, user, security, and compliance
requirements.
🪄 Autofix (Beta)

✅ Autofix completed


ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 6772e8c2-fba8-4011-893a-ecd127f437d6

📥 Commits

Reviewing files that changed from the base of the PR and between 686ea39 and f1dbbc4.

📒 Files selected for processing (1)
  • AGENTS.md

Comment threadAGENTS.md Outdated
Comment threadAGENTS.md Outdated
@coderabbitai

coderabbitaiBot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Note

Autofix is a beta feature. Expect some limitations and changes as we gather feedback and continue to improve it.

Fixes Applied Successfully

Fixed 1 file(s) based on 2 unresolved review comments.

Files modified:

  • AGENTS.md

Commit:57bfe6567b1390449b842359c5b0e3c1b2741089

The changes have been pushed to the claude/frontend-checklist-skills-ece5e6 branch.

Time taken:3m 30s

Fixed 1 file(s) based on 2 unresolved review comments.
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
@BigSimmo
BigSimmo merged commit 3e65844 into mainJul 28, 2026
19 checks passed
@BigSimmo
BigSimmo deleted the claude/frontend-checklist-skills-ece5e6 branch July 28, 2026 03:40
BigSimmo added a commit that referenced this pull request Jul 28, 2026
Squash-merged; remote branch auto-deleted. Records the CodeRabbit outcome
(2 of 3 findings autofixed pre-merge, third landed in #1308) and that
verify:cheap never completed past check:installed-lock-parity.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@BigSimmo