Skip to content

[auto] Add principle-no-lookahead, principle-report-the-disqualifier, report-rendering - #300

Open
EdbertChan wants to merge 2 commits into
mainfrom
reflect/no-lookahead-20260909
Open

[auto] Add principle-no-lookahead, principle-report-the-disqualifier, report-rendering#300
EdbertChan wants to merge 2 commits into
mainfrom
reflect/no-lookahead-20260909

Conversation

@EdbertChan

@EdbertChanEdbertChan commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Summary

Two principles from a backtesting session that spent roughly three hours producing results from a signal that could not have been known at the time it was credited to.

principle-no-lookahead — score a decision only against inputs that existed before that decision was made. Names the one defect across its four vocabularies (leakage, look-ahead bias, point-in-time correctness, repainting), and carries the cheap mechanical test: recompute from only the data available at time t and compare against the full-history computation; any difference is a leak.

principle-report-the-disqualifier — a report is finished when a reader who disagrees knows where to look. Requires a falsification observation, a measured/inferred/conjecture grade per claim, ruin reporting where the path can cross an absorbing barrier, and the count of configurations tried.

Review Claim

Both skills state a general principle, name the established prior art with DOIs, and carry a mechanical test rather than only prose.

Safety Invariant

Where no prior art exists it is recorded as such rather than invented: "repainting" is marked vendor and forum vocabulary with no peer-reviewed source.

Motivation

The incident: a study's chart view placed a pivot on the bar it occurred; the live view only from its confirmation bar, 15 to 50 minutes later. A truncation test showed the signal at 00:05 does not exist when the data ends at 00:05, 00:30 or 00:45, and appears when it ends at 00:55. Backtesting the chart view produced 23 green days out of 23 and 38,970% on a 10,000 USD account; the live view liquidated the same account. Neither the too-good result nor the unvalidated code path had a name in the corpus.

Non-goals

No existing skill, hook, or install path is modified. No always-on file is touched.

Test Plan

Manual review. Both files carry valid frontmatter; every claim of prior art resolves to a DOI or a stated absence.

Revert Plan

git revert the commit; two new directories, no existing file changed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01F43CBUnsDEs6J2zEC1r8a8


Note

Low Risk
Adds markdown skill docs under corpus/skills/ only; no runtime, auth, or data-path changes.

Overview
Adds three new corpus skills (documentation-only; no hooks or install paths changed).

principle-no-lookahead codifies point-in-time evaluation: separate display vs decision artifacts, require cutoffs and a truncation test (prefix recompute vs full history), treat implausibly good metrics as bugs, and cite leakage/rolling-origin literature—with explicit “no peer-reviewed prior art” for repainting vocabulary.

principle-report-the-disqualifier defines what causal or outcome reports must expose: falsifying observations, measured / inferred / conjecture grades, absorbing-state (“did the path survive”) columns, before/after fix headlines, and sensitivity to frozen constants; it cross-links to existing assert-invariants and no-lookahead skills.

report-rendering requires delivering analyses as one self-contained HTML (embedded data: figures, print CSS) plus PDF from the same HTML via headless Chrome, with a minimal Python + shell recipe and a gate to not render unvetted numbers.

Reviewed by Cursor Bugbot for commit bff693d. Bugbot is set up for automated code reviews on this repo. Configure here.


Added: report-rendering

Third skill on this branch, from the same session. A three-section analysis was
called finished while it sat as separate markdown files in a repo; asked to see
the report, the only answer was a list of paths.

Ship one self-contained HTML file with figures embedded as data: URIs, plus a
PDF printed from that same HTML so the two cannot disagree. The recipe needs
nothing beyond a browser: markdown to HTML, then
google-chrome --headless --print-to-pdf. Print styles are called out
explicitly because their absence is invisible in the browser and only breaks in
the PDF.

Verified by producing reports/master-report.html (885,056 bytes, figure
inline) and reports/master-report.pdf (4,477,296 bytes).

Grounding: RFC 2397 for data: URIs, Knuth 1984 and Peng 2011 for the
document-and-analysis-as-one-artifact lineage, and an explicit "no known prior
art" for the delivery-format principle itself.

Problem: a backtesting session produced results for roughly three hours from a
signal that could not have been known at the time it was credited to, and the
test written to validate it asserted against a helper built to make that test
pass rather than against the path every consumer read. Nothing in the corpus
named either failure, so neither was catchable.
Considerations: two skills rather than one, because the failures are separable.
The first is about which inputs a decision may read; the second is about what a
report must disclose once the evaluation is clean. Both carry the mechanical
test rather than only prose: the truncation replay for lookahead, and the
required Falsification and Confidence sections for the report. Each cites the
literature that names the failure, and where none exists the entry says so
rather than inventing a lineage: "repainting" is recorded as vendor and forum
vocabulary with no peer-reviewed prior art.
Blast radius: two new skill directories under corpus/skills. No existing skill,
hook, or install path is modified.
Verification: both files carry frontmatter with name and description; the
lookahead skill cites Kaufman et al. TKDD 2012 for the formalisation of
leakage, Tashman IJF 2000 for rolling-origin evaluation, and Bailey et al.
Notices AMS 2014 for backtest overfitting; the disqualifier skill cites Popper,
Chambers and Tzavella 2022, Guyatt et al. BMJ 2008, Peters 2019 and Kelly 1956
for the absorbing-barrier case, and Simonsohn et al. 2020. Prose and citations
were recovered verbatim from the reflect subagent transcript rather than
rewritten.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F43CBUnsDEs6J2zEC1r8a8
@cursor

cursorBot commented Sep 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_460a78d6-0366-4542-b122-182afa359448)

Problem: a three-section analysis totalling 220 KB of markdown was described as
finished while it sat as separate files in a repo. Asked to see the report, the
honest answer was a list of paths. The reader either lacks a checkout or must
open several files in the right order and imagine the figures in place.
Considerations: the recipe deliberately requires nothing beyond a browser,
because a rendering step that needs pandoc or LaTeX gets skipped. Figures embed
as data URIs so the file survives being moved or emailed, which a relative path
does not. Print styles are called out separately because their absence is
invisible in the browser and only shows up in the PDF, where sticky table
headers repeat on every page.
Blast radius: one new skill directory under corpus/skills. No existing skill,
hook, or install path is modified.
Verification: the recipe as written produced reports/master-report.html at
885,056 bytes with the distribution figure inline, and
reports/master-report.pdf at 4,477,296 bytes via
`google-chrome --headless --print-to-pdf`, in the trend-reversal repo. The
md.reset() note comes from the toc extension carrying anchors between sections
during that run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F43CBUnsDEs6J2zEC1r8a8
@cursor

cursorBot commented Sep 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_c05823a5-0592-482b-9cf1-b516cb3b7784)

@EdbertChanEdbertChan changed the title [auto] Add principle-no-lookahead and principle-report-the-disqualifier[auto] Add principle-no-lookahead, principle-report-the-disqualifier, report-renderingSep 9, 2026
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@EdbertChan@edbert-bot