[Bench #478] Benchmark every Flow-implemented estimator operation, not only the 19 headline rows

[godofecht]

Parent: #478

The architecture inventory contains 491 sklearn estimator-operation rows, while the canonical headline intentionally covers only 19 estimator/dataset combinations. Any Flow-implemented estimator or operation outside that set is currently capable of hiding a severe regression.

Build an implementation-surface registry and require every public Flow estimator operation to have at least correctness/parity smoke, representative warm timing, scaled timing classification, memory evidence tier and execution-substrate attribution. Unsupported sklearn operations should be distinguished from implemented-but-unbenchmarked operations.

Validation is local-only. Do not add, enable, trigger, inspect, wait for, or rely on GitHub Actions or other hosted CI/workflows. Coverage ratcheting should be enforced by a deterministic local command from a clean checkout and recorded in machine-readable artifacts keyed to the exact source SHA/environment.

Acceptance: implemented public operations with no benchmark evidence = 0; coverage is machine-readable and locally ratcheted; the 19-row suite remains the stable headline subset, while the full surface produces a broader wins/ties/losses dashboard and new losses become focused issues.

[godofecht]

I worked through the implementation-surface registry design as a concrete coverage ratchet.

The useful split is public-operation evidence versus reusable implementation-level evidence. Let S be public implemented estimator operations and φ:S→I map each operation to a semantic implementation-equivalence class. Each public op keeps its own correctness/parity smoke, while expensive warm/scaled/memory/substrate evidence is keyed to φ(o). Coverage is

Covered(o) = Smoke(o) && ((H[φ(o)] & R[o]) == R[o]).

This gives two properties we want simultaneously: a new public API operation cannot hide because it must get operation-local smoke evidence, while aliases/wrappers that truly lower to the same numerical/representation/substrate contract do not need duplicate heavy benchmark records.

Under the rule that heavy evidence may only be reused within a valid semantic implementation class, one heavy evidence record per represented implementation class is necessary and sufficient. Evidence growth is monotone: adding evidence cannot make an already-covered operation uncovered; only API growth, implementation-identity changes, or stronger evidence requirements can trip the ratchet.

I encoded the model and the local validator cases in Flow and archived the full derivation in OpenProjects:

• ml/implementation-surface-benchmark-ratchet.md
• ml/implementation-surface-benchmark-ratchet.flow

The fixed experiment has 8 public operations backed by 5 implementation classes, proves an added alias can reuse heavy evidence after its own smoke passes, proves a genuinely new implementation fails coverage until a new evidence record exists, and proves requirement strengthening fans out exactly to users of the deficient implementation class. I also independently exhaustively checked monotonicity over the complete 4-bit evidence lattice locally.

A production implementation_surface.json can therefore remain operation-complete without turning the 19-row headline benchmark into the coverage mechanism. The next useful measurement is |S|/|I| on the real Flow API: that tells us how much expensive benchmark work can soundly collapse through shared implementation identities.