docs(results): how much extraction leaves on the table — greedy DP vs exact

[jppittman]

Three independent budget-loosening changes each improved most real kernels and regressed a minority — #1101 (numeric-first rule order, 146 up / 21 down), #1109 (removing the class-cap break, 140/16), #1114 (live-counted class budget, 155/4, +2.03% for 2.00x applications). The mechanism named in #1114 and established in #1111 is that extract_dag is a greedy DP over a static prior and is not argmin (§1.L5 of docs/plans/2026-09-02-optimizer-api.md), so a larger e-graph is not monotonically better even though the optimum over it can only fall.

This PR answers the question that sequences those trades: how much is the greedy chooser leaving on the table? Measurement + docs only — no production behavior changes.

Method

Two references, both under Optimizer::production() and CostModel::latency_prior() — the same cost function the DP uses — so the comparison is chooser-vs-chooser, never cost-model-vs-cost-model. 302 kernels: 206 real .arena dumps + a 96-point synthetic size ladder.

1. Knuth's algorithm (Dijkstra generalized to AND-OR graphs) — the exact minimum tree cost, in polynomial time, on every kernel, no time limit. That is the objective extract_dagnames, so it separates "the DP fails its own objective" from "the objective is the wrong one".
2. Branch and bound for the true DAG optimum (each chosen class priced once) with dominance filtering, an admissible chain lower bound, and reduced-cost fixing. NP-hard, so it is budgeted and reports UNSOLVED rather than a truncated answer; cross-checked against brute-force enumeration on every instance small enough to enumerate.

Headline

reference	scope	pooled recovery vs greedy	per-kernel
tree optimum (exact, polynomial, no limit)	all 302, nothing excluded	0.258% (real-only 0.284%; shaders 2.62%)	median 0.000%, p90 0.403%, worst 14.557%; 68 kernels strictly improved
DAG optimum (4s cap)	89 of 302 proved	0.228% on those 89	greedy already exactly optimal on 86 of 89; worst ratio 1.1212
certified floor (cheapest term the search exhibited)	all 302	0.286%	strictly beats greedy on 81

Caveats that travel with the number: it is against the tree optimum; exact DAG extraction stops being computable at ~100–200 reachable e-classes while a production glyph saturates to a median 1,755, so 213/302 are UNSOLVED and excluded from the exact statistic; and all 68 kernels the tree optimum beats are real (64 glyph, 4 shader) — zero synthetic — so the two references describe different populations.

Mechanism split — which of the three to fix

#	defect (#1111 / §1.L5)	share of proved loss	what fixing it recovers	cost
(i)	single DFS, not a fixpoint (CYCLE_COST, never revisited)	10/195 (5%)	0.258% pooled, all 302, p90 0.403%, worst 14.557%, 68 kernels — and both U+004B regressions	small
(ii)	tree cost, not DAG cost (sharing unpriced)	185/195 (95%)	0.028% pooled beyond the tree optimum	NP-hard
(iii)	total_cost read at extract.rs:1636 before repair at :1639	0	no cycles at all	trivial

The 95/5 split is a small-kernel statement — it is computed only where the branch and bound closed. Over the full corpus the picture inverts, and (i) is both the larger measurable recovery and the cheaper fix. Cycle-priced classes are not rare: 238/302 kernels have at least one (median 10, max 284), and on 203/302 the best term the search holds uses one.

(iii) is worse than "stale": of the 132/302 mismatches not one is an ordinary-magnitude discrepancy. 92 report exactly usize::MAX (the .unwrap_or at :1636, taken when the DFS never resolves the root class) and 40 report 1x, 2x or 3x usize::MAX / 4 — the Dwrt sentinel from cost.rs:292, summed into a number typed as a cost. No shipped result is corrupted by it — #1101/#1109/#1114's harnesses re-score the returned arena (arena_static_cost, runtime.rs:1278) and guide_headroom.rs:395's 800-expression corpus holds no sentinel — so it is a live trap, not a live wound.

The finding that settles the sequencing

On glyph16:U+004B and glyph32:U+004B — two of #1114's four regressions — the roomier e-graph provably contains a term costing 1013 while greedy returns 1132 on it and 1047 on the smaller production graph. The graph got strictly better and the chooser gave up 8.1% anyway: those regressions are extraction failures, not budget failures.shader:julia_set leaves 5.6% on the table at the production budget with no budget change in play. psychedelic (766 → 816) exhibited no cheaper term inside the limit — unresolved, not exonerated.

Sequencing

Fix extraction before taking #1101, #1109 or #1114. The fixpoint is free at run time, recovers more on real kernels than #1114 buys for 2x the rule applications where they overlap, and turns #1114's U+004B regressions into improvements — which no budget tuning can do.

Cost of a fixed extractor (estimate, not benchmarked): Knuth is O(Σ arity · log V) against the DFS's O(Σ arity) — a small constant and a log, not an order — and it completed on all 302 kernels including the 4,847-class psychedelic. Extraction runs once per compile; saturation runs 8,729,067 rule applications over 800 expressions in the guide-headroom corpus (median 194.5, p90 31,197, max 996,047), and #1114 doubles that for +2.03%. This is the cheap place to spend.

Shape

• Probe in its own file, #[cfg(test)] + #[ignore]d — runtime.rs was already 2,631 lines and flagged for size.
• The shared .arena loader moved out of runtime.rs into arena_corpus (net −124 lines there), so the two probes that read the corpus do not carry two copies of the dumpers' inverse.
• No public API added; no production path touched.

Results: docs/results/2026-09-02-extraction-gap.{md,csv,json}.

Gates: build, cargo test --workspace, clippy --workspace --all-targets -D warnings, fmt --check, and both no_std checks (pixelflow-ir and pixelflow-search, --no-default-features) all green locally.

Refs #1111, #1101, #1109, #1114.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

[github-actions]

🤖 Hi @jppittman, I've received your request, and I'm working on it now! You can track my progress in the logs for more details.