Make Redgate the default interactive workflow - #82
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
It updates skill prose but (per repo guidance) should include a PR demonstration comment for the skill change, and it also contains a confirmed documentation typo in the criteria-contract frontmatter.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR makes the redgate plugin the default router for Jordan’s nontrivial workflows and standardizes an interactive “ask-question” contract for all user decisions, while preserving T0 direct handling to avoid unnecessary ceremony.
Changes:
- Updates Redgate/criteria-contract prose to auto-trigger on nontrivial work and enforce interactive, tap-ready decision prompts (including MAJOR-gate confirmations).
- Bumps the Redgate plugin metadata/version to
0.1.0and refreshes marketplace + example gallery descriptions to match. - Adds deterministic “cheap” eval checks to ensure the new routing and question contract language remains present.
File summaries
| File | Description |
|---|---|
| plugins/redgate/skills/redgate/SKILL.md | Adds default-routing + interactive question contract rules and MAJOR-gate tool confirmation language. |
| plugins/redgate/skills/redgate/references/calibration.md | Aligns calibration guidance to interactive, one-decision-per-call questioning under a shared budget. |
| plugins/redgate/skills/criteria-contract/SKILL.md | Updates ARM interview procedure prose to require the interactive ask-question tool; contains a run-directory path typo (see comment). |
| plugins/redgate/evals/cheap/checks.sh | Adds cheap-tier assertions that the default-routing + question contract language remains present. |
| plugins/redgate/commands/redgate.md | Updates command description and instructions to match the new interaction contract. |
| plugins/redgate/AGENTS.md | Adds default operating mode + interactive question requirements for harnesses reading AGENTS. |
| plugins/redgate/.claude-plugin/plugin.json | Bumps plugin version and refreshes plugin description to the new default-router framing. |
| docs/examples/index.html | Updates the redgate skill description in the generated examples gallery. |
| AGENTS.md | Adds repository-level guidance: auto-trigger redgate for nontrivial work and require interactive ask-question decisions. |
| .claude-plugin/marketplace.json | Updates marketplace entry for redgate (version + description). |
Review details
- Files reviewed: 10/10 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
JRichlen
left a comment
There was a problem hiding this comment.
Independent architecture + integration review
1. Redgate needs explicit composition precedence, not just broad auto-triggering
The new root/plugin guidance makes Redgate the default router for planning, research, design, implementation, debugging, refactoring, review, deployment, multi-agent coordination, etc. That set overlaps heavily with specialist plugins already in the marketplace (codebase-design, diagnosing-bugs, orchestrate, wayfinder, scope-fence, and others).
The intended architecture appears to be Redgate as the protocol/gating envelope, not Redgate replacing specialist routing. Please make that precedence explicit before merge. Suggested semantic rule:
Route to the most specific applicable skill(s) for domain behavior; use Redgate as the execution/verification envelope when the work is nontrivial enough to require rounds, explicit criteria/evidence, or a classified human gate.
Otherwise a router can reasonably read the new broad Redgate description and choose only redgate for almost every meaningful task, defeating the specialist trigger roster.
This matters directly to Agent OS PR #83, which currently treats redgate as the verified-execution/gating layer and grill-me/other skills as separate composable capabilities. The two PRs should encode the same boundary.
2. Preserve the UI contract as a portable capability
The structured ask-question behavior is useful, but the durable invariant should remain behavioral: use the harness-native structured choice/confirmation primitive when available, with compact textual options as the fallback. Avoid wording that makes a specific tool/API name part of the cross-harness contract.
3. Existing Copilot findings remain blockers for a clean re-review
Copilot already identified the .redgate/slug/ vs .redgate/<slug>/ typo and the repository-required demonstration comment for skill changes. Those should be resolved before treating the new review as final.
Verdict
Direction is sound, but I would tighten the Redgate-vs-specialist composition rule before merge. That prevents the new default from becoming a routing black hole while preserving the intended default verification discipline.
There was a problem hiding this comment.
🔵 Needs a closer look
The criteria-contract skill frontmatter currently contains an incorrect .redgate/slug/ path that contradicts the rest of the protocol/docs and should be corrected before merge.
Review details
Suppressed comments (1)
plugins/redgate/skills/criteria-contract/SKILL.md:6
- The frontmatter description says the run directory is
.redgate/slug/, but the rest of this skill (and the scaffold script/docs) consistently use.redgate/<slug>/. This typo will mislead users and also propagate into generated docs that read the description from frontmatter.
The ARM stage of Red Gate: turn an interviewed idea into CRITERIA.md and
check.sh in .redgate/slug/, prove the gate red (every checkable
criterion FAIL, harness preflight clean), get human ratification, and pin
- Files reviewed: 12/12 changed files
- Comments generated: 0 new
- Review effort level: Lite
|
@claude I want to use this PR as a collaborative architecture refinement, not just a code review. Please review the current PR plus PR #83 and challenge/refine this proposed boundary:
Please specifically pressure-test:
Treat existing Copilot findings as inputs, not the whole review. Prefer a concrete proposed contract / wording over abstract commentary. When you respond, please include |
|
To use Codex here, create a Codex account and connect to github. |
|
@claude One correction to the cross-PR framing before you review this: Redgate and Agent OS serve different purposes and neither should subsume the other.
That means #82 should avoid language implying Redgate is the universal router for all nontrivial work. A better contract is:
Likewise, Agent OS does not need to inherit its interactive curation UX from Redgate. Both may use the same harness-native structured-question capability and both may compose Please refine #82 around this harness-layer identity and identify the smallest trigger condition for Redgate that avoids both extremes: universal ceremony vs explicit-only invocation. Concrete wording preferred. Include |
|
To use Codex here, create a Codex account and connect to github. |
There was a problem hiding this comment.
🟡 Changes recommended
The examples gallery HTML is not consistent with the current Redgate SKILL frontmatter description (and will likely fail the cheap-tier docs/build-examples.sh --check), and there’s a small grammatical error in plugins/redgate/commands/redgate.md.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
- Files reviewed: 12/12 changed files
- Comments generated: 2
- Review effort level: Lite
|
@JRichlen Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub. You can ask me to try again later by mentioning me in a new comment. If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: Sorry for the inconvenience! |
Co-authored-by: JRichlen <9574264+JRichlen@users.noreply.github.com>
|
@codex Please address RQ-003 only on this existing PR branch ( Current head Cheap tier
Diagnose whether #2 is stale assertion wording versus an actual regression against the PR's intended current architecture. Preserve the intended boundary: specialist owns domain procedure; Redgate is the evidence/verification/gating envelope when warranted. Do not simply weaken or delete the check. Routing tierPromptfoo ran 50 cases: 46 pass, 1 semantic failure, 3 errors.
Use Redgate discipline: pin the branch-local criteria, reproduce the current failures, make only the smallest changes needed for this PR's declared semantics, then rerun relevant cheap + routing checks and attach exact evidence. Preserve the real demonstration-comment requirement rather than fabricating a demo. Push fixes to this existing remote PR branch. Completion requires updated remote commits and fresh CI evidence. Do not merge automatically. |
Summary
Testing
|
…Q-003 Two cheap-tier failures and one routing semantic miss at head 80f2734: - Sync the redgate marketplace.json description to plugin.json's specialist-first framing (working harness / verification envelope, composed around the most-specific specialist skill), which the branch adopted but only in plugin.json. Regenerate evals/routing/roster.txt from the synced metadata. - Replace the stale "common workflow trigger roster" cheap check with three architectural checks matching the PR's declared semantics: the driver frames Red Gate as the default verification envelope, states specialist-first routing precedence, and states the ceremony bypass for specialist work with no evidence contract or classified gate. The old check asserted the universal-router roster prose this branch deliberately removed; the replacements are strictly stronger than deleting it. - Clarify the routing prompt: a request that merely contemplates the governed act ("should I just write one?") routes the same as the act itself. Targets the debounce semantic miss (routed none instead of the search-before-writing discipline) without changing the expected route, the roster contract, or the pass-rate floor. Transport/empty -output FAULT handling in evals/paid/pass-rate.sh is untouched and already distinct from semantic failures. Evidence: evals/cheap/run.sh 1175 passed, 0 failed (was 1171/2); promptfoo validate on evals/routing/promptfooconfig.yaml is clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsPQTo67oovz8ogirsH6uL
… arbiter The routing job on 317a05b recorded 40 pass / 0 semantic failures / 10 errors — every error an OpenRouter 504 ("The operation was aborted"), i.e. the transport-FAULT class pass-rate.sh exists to handle. Two defects made that class fatal anyway: - promptfoo only retries 5xx responses when PROMPTFOO_RETRY_5XX is set (verified in the 0.122.0 source: fetchWithRetries throws-and-retries on >=500 only behind that flag), so each 504 became a row error on the first hit. Set it in the eval step's env. - The eval step ran promptfoo bare, so its exit 100 (any red row, errors included) killed the job before the statistical gate ever ran. The gate is the designed arbiter: it excludes FAULTs, enforces the 0.8 per-scenario floor over VALID samples only, and fails closed when FAULTs starve a scenario (min-runs 3). Tolerate promptfoo's exit as long as results.json exists; the gate step keeps the verdict. The floor, min-runs, and fail-closed semantics are untouched — a semantic miss or a FAULT-starved scenario still reds the job. Evidence: cheap tier 1175 passed, 0 failed; branch-protection drift guard green (no job renamed, pass-rate.sh still wired). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsPQTo67oovz8ogirsH6uL
|
RQ-003 complete on this branch, in two pushed commits: Cheap tier
Evidence: Routing tier
Fresh CI evidence (run 33336710469, routing job, head Zero semantic failures; every red row an explicit 504 FAULT, excluded per the declared semantics. ScopeNothing absorbed from #87/RQ-001 or #88/RQ-002. Diff: Not merging — leaving that to you. Generated by Claude Code |
|
Demonstration comment (required by the repo's demonstration discipline — this PR edits skill prose: the redgate driver The skill change applied to real inputInput: the 10 labeled routing requests in Before (head After (head The branch also carries the skill applied to its own construction: the The misses
Generated by Claude Code |
…a 7) The boundary reframing merged as e2a53a9 was a real turn of the trajectory and belongs on the page: the same PR that set out to make Redgate the default interactive workflow first crowned it the universal router, then corrected itself in flight — a specialist owns the domain procedure; Redgate is the verification envelope around how it is executed. Recorded as a new era 7 ("The gate learns its place") since era 6's lesson is about consumable proof surfaces, not routing. The skim heading was hardcoded "The story in six lines"; it now derives from the era count so curation edits can't leave it stale. Regenerated index.html; build-timeline.sh --check passes byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsPQTo67oovz8ogirsH6uL
…(docs only) The docs half of #83, split out per review: the lousy-agents handoff and the automation-curation lens, with the redgate suggested-skills line and Decision framing synced to the merged #82 boundary, the Adapter+ multiplicity legend added, and the experiment-evidence link pinned to #83's branch. The experiment harness stays on #83 under #85's guardrail.
- The thesis no longer claims every era was forced by a recorded failure; some were forced by a gap the previous answer left open. It names the second question the story keeps answering (who is allowed to do what) and says that eras are threads, not calendar slots, since several overlap by a day. - Era 3's bridge no longer says "no one had yet asked the field": the 88-pattern corpus was written an hour after the protocol. - Era 4's bridge points back to that verdict instead of dating it to the end of era 4. - Era 6's bridge no longer credits the published surfaces with exposing the router overreach; the maintainer's review on PR #82 did, within the hour. - Era 7's bridge, era 8's lesson, the Act III summary and the honest-gate entry now state the real link the page was hiding: the fail-open statistical gate was caught while the routing tier's live runs were being scored (pass-rate.sh records it as observed on PR #93). - Era 1's lesson mentions the five plugins that shipped in July. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014XYwiWZicXu2CfDUKAXHpi
What changed
.redgate/<slug>/path typo found by CopilotCross-PR boundary with #83
executionPolicy: redgategrill-me, but neither owns the other's UXVerification status
Previous branch verification reported Codex skill validation and the repository cheap tier green before the latest architecture refinements. Fresh CI/review is required on the current head before merge.
Copilot's earlier demonstration-comment requirement remains intentionally unresolved until a real Redgate demonstration is actually run; do not fabricate one.