M1: NFT tracker parity — tracking issue

[kalwalt]

Goal

Deliver milestone M1 — parity of the NFT tracker decided in ADR-0001: a detection-only NftTracker in packages/nft-tracker that does exactly what the stateless webcam demo does today, plus the .wnft target format (spec) readable by two independent codecs.

M1 adds no new tracking behaviour. Its job is to give every later milestone (M2 patch tracker, M3 IPPE + One Euro, M4 target compiler) a tested baseline to be measured against.

Why this issue exists

Accepted ADRs are not edited (see AGENTS.md, "Design records"), so ADR-0001's action items record the plan at acceptance time. Progress is tracked here instead. The M1 items of the ADR are 4, 5 and 6; item 7 (M2–M4) is out of scope.

Checklist

Done

• ADR-0001 and the target format spec accepted — docs: add ADR-0001 and NFT target format spec v0.1 #20
• Scaffold packages/nft-tracker and the in-memory target types; spec 0.1 rev 2 — feat(nft-tracker): scaffold the package and the in-memory target types #21

Spec

• Decide Q10 (I-JSON): require the manifest to be I-JSON (RFC 7493), rejecting duplicate keys? Must be settled before 3A, since the TS codec is where duplicates would be detected.
• Untick ADR-0001 action item 3 (it was ticked in feat(nft-tracker): scaffold the package and the in-memory target types #21); from now on progress lives in this issue. Fold into the Q10 PR.

Implementation

• 3A — TS codec and fixtures. encode / decode in packages/nft-tracker/src/target/format/, validation order of §6.1, error and warning codes of §6.2, canonical writer of §7.3. Fixtures in fixtures/nft-target/0.1/ (valid/, invalid/, warnings/, noncanonical/) from a committed deterministic generator. Tests of §8.2–§8.4, including truncation and property-based fuzzing.
• 3B — NftTracker, parity. Move buildLevelIndex / matchPerLevel from examples/js/pinball-shared.mjs into the package (zero-copy per-level views); buildFromImage; detection-only NftTracker; parity test against the static demo pipeline with a seeded RNG; demos import from the package. Can run in parallel with 3A.
• 3C — Rust codec crates/wnft-format. Written from the spec, not ported from TS; no_std + alloc, no unsafe, no panic on untrusted input. Consumes the 3A fixtures, never generates its own. Adds a Cargo workspace and a Rust CI job. Starts after 3A is merged.
• 4 — Integration and first real target. End-to-end round-trip test (image → buildFromImage → encode → decode → NftTracker); compile-target script; commit examples/targets/pinball.wnft; static demo can load it. If 3C is merged, a Rust test decodes pinball.wnft too. Needs 3A and 3B.

  Blocked by cv-backend-jsfeatnext: detect is not stateless — FAST corner rows read uninitialised cache memory (off-by-one) #27. pinball.wnft is built through detect, so
  generating it now would commit a file whose keypoints depend on
  uninitialised memory — irreproducible, and the first real target the
  Rust codec is meant to decode. 3C is not blocked: the format fixtures
  are synthetic and never touch detect.

Measurement

• 5 — Benchmark page. examples/bench-nft.html: per-stage timings (frame acquisition, grey conversion, detect, describe, match, homography, pose), p50 / p95 / max, JSON export; stateless and NftTracker modes.
• Pick the reference device (a mid-range Android phone) and commit the first baseline JSON. ADR-0001 point 5 thresholds are evaluated against it.

Contract

• 6 — File the cv-backend-spec issues listed under "Contract gaps" in ADR-0001, one each:
  • Declare the pyramid scale step (Keypoint.level is ambiguous across backends)
  • Cross-backend descriptor conformance ("same DescriptorKind, different bits")
  • k-nearest matching (prerequisite for multi-view descriptors, WKNF_multiview)
  • Detector selector and spatial distribution in DetectOptions; estimator selector in RansacOptions
  • Planar pose candidates (poseFromHomography returns a single pose)
  • Injectable RNG for estimateHomography — cv-backend-spec: injectable RNG for estimateHomography (RansacOptions.rng) #24
  • detect is not a pure function of (image, options) — uninitialised
    buffer reads in jsfeatNext's FAST — cv-backend-jsfeatnext: detect is not stateless — FAST corner rows read uninitialised cache memory (off-by-one) #27

Dependencies

Q10 ──► 3A ──► 3C
         └──┐
 3B ────────┴──► 4 ──► 5 ──► reference-device baseline
             #27 ──┘
 6 (independent; #24, #27 filed)

Definition of done

• NftTracker in detection-only mode produces the same result as the stateless pipeline on the static demo images, with a seeded RNG.
• pinball.wnft is produced by compile-target, and decoded identically by the TS codec and by crates/wnft-format.
• All fixture categories of spec §8 pass in both codecs; CI runs npm and Rust jobs.
• A baseline JSON from the reference device is committed.
• The five contract issues are open and linked here.