Uh oh!
There was an error while loading. Please reload this page.
docs(e2e): Chapter 2 exploration runbook — kick-off, triage, promotion (#401) - #445
Conversation
…n pipeline (#401) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Self-review catch: task-8-report.md lives under .superpowers/ (gitignored), so pointing the shipped runbook at it as evidence was a dead reference for anyone without that local planning history. Describe the acceptance-testing evidence inline instead, and fix "earlier round of the same run" to the accurate "separate run of the same acceptance round" (task-8-report.md's run 4 found the tutor-resume 500; run 5 found the wrong-data bug). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deploying with |
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs | frontend-staging | df2672b | Commit Preview URL Branch Preview URL | Jul 28 2026, 10:57 AM |
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughAdds a complete Chapter 2 exploratory-testing runbook covering execution modes, prerequisites, artifacts, oracle validation, finding triage, regression promotion, locking, cadence, and troubleshooting. Links the runbook from the local Supabase documentation. ChangesExploratory testing runbook
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related issues
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
AndresL230
commented
Jul 28, 2026
Code reviewFound 1 issue:
Sapling/docs/e2e-exploration.md Lines 175 to 190 in 3c65b97 🤖 Generated with Claude Code - If this code review was useful, please react with 👍. Otherwise, react with 👎. |
…iming, cross-refs (#401) Code-review on PR #445 found the flagship "wrong-data upload" example was a seam artifact (function-mode's E2E_DOC_* fixtures return identical canned output for every upload by design), not a real bug — reframe it as a worked "drop + improve the harness" triage example and promote the genuinely novel tutor-resume 500 (concept_describe unregistered in agents/function_handlers_e2e.py) to the flagship promotable example instead. Also: reconcile the "few minutes" (§3) vs "~10 minutes" (§9) timing claims into one warm-cache-vs-first-run story, and drop two dangling section cross-refs (§6's inline traces/ caveat didn't need a pointer; "forces both (see §3)" pointed at a section that never explained the fact) plus align the --check example with the oracle's own sorted default order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
#401) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AndresL230
commented
Jul 28, 2026
Review finding fixed in 56511af: the promotable worked example is now the genuine tutor-resume 500 (concept_describe unregistered — facts re-verified against _providers.py/graph.py/main.py by the re-review), and the wrong-data episode is recast as a seam-artifact triage lesson with the recognition tell. Timing contradiction and dangling cross-refs fixed in the same commit; a code-span rewrap nit fixed in the follow-up. Scoped re-review: all findings addressed. |
Chapter 2 of the E2E program (epic #403), PR 3 of 3. Closes#401.
The operator's manual for
make explore//explore(PR #444) and the oracle CLI (PR #443): what Chapter 2 is (vs Chapter 1's scripted journeys — local-only, never gates), prerequisites, both kick-off modes with the full knob table, the.explore/output inventory, standalone oracle usage, the triage protocol forfindings.md(reproduce → issue with evidence; noise → tune the oracle; known-bug → evidence only if new), the promotion pipeline (issue → scripted Chapter 1 journey, testids per docs/frontend-testids.md; epic bar: ≥3 findings promoted into Chapter 1 regressions in the first month), cadence/cost, the machine-singleton lock protocol, and troubleshooting.Written against the MERGED harness (post-review reality): unconditional dummy
GEMINI_API_KEY+SEED_RICH=1,Edit(.explore/**)-scoped explorer writes,EXPLORE_USERfive-user wiring, holder-writtenlock.okbookkeeping, session recordings under.explore/traces/. Every command, knob, default, path, and exit code in the doc was verified against the code (knob table cross-checked with/usr/bin/grep -o 'EXPLORE_[A-Z_]*', check names againste2e_oracles.__main__.CHECKS,make -ntargets).Plus one pointer line in docs/local-supabase.md's "One-command E2E stack" section.
🤖 Generated with Claude Code
Summary by CodeRabbit