feat(gradcheck): finite-difference gradient checker + OpInfo registry (ADR 091, T1.1)

[dndungu]

Implements T1.1 of the GPU training-stack hardening plan (zerfoo docs/plan-gpu-training-hardening.md), the gradcheck leg of zerfoo ADR 091. Refs #128.

Package location: testing/gradcheck

Chosen over graph/gradcheck because the checker is test infrastructure, not graph runtime: it sits beside the existing testing/testutils, and it must import compute (for the CPU engine) in addition to graph -- keeping it out of graph/ keeps the graph package free of a compute-engine-facing test dependency and avoids any future cycle when graph nodes want to import the checker in their own tests.

Checker math

For a node f with inputs x_j and parameters p_j:

• Forward + analytic Backward(FullBackprop, g, inputs...) run at float64 on the CPU engine (precision first; GPU is a separate harness).
• Upstream gradient g: caller-supplied, or deterministic pseudo-random entries with magnitude in [0.25, 1.0] and random sign (randomized upstreams catch transposed/structural Jacobian errors an all-ones upstream can mask).
• Numerical gradient per element by central differences: h = 1e-6 * max(1, |x|), num = sum_i g_i * (f(x+h)_i - f(x-h)_i) / (2h).
• Comparison per element: mismatch iff |a - n| > atol + rtol * max(|a|, |n|) with defaults atol 1e-7 / rtol 1e-5 at f64; per-op overrides via OpInfo.Tol/OpInfo.Eps.
• Fresh node instance per evaluation: MakeNodeFn constructor closure is invoked for the analytic pass and for every perturbed Forward, so Forward-cached state (softmax output, layernorm statistics -- the class behind the GPU LayerNorm cached-variance bug) can never leak between evaluations. Parameter values are snapshotted from a reference instance and copied into each fresh instance, so randomly initialized constructors are fine as long as parameter order/shapes are deterministic.
• Parameter gradients are read from Parameter.Gradient after Backward (the accumulation convention), zeroed beforehand.

Registered ops (26)

ztensor's graph package ships no public op nodes (only unexported inputNode/checkpointNode plumbing), so the registry covers graph.Node wrappers around the compute.Engine operations training graphs are built from -- the wrappers' backwards are themselves composed of engine ops/kernels, so the engine is what gets exercised:

• Elementwise binary: Add, Sub, Mul, Div, Pow
• Activations / unary: Tanh (backward via the fused TanhPrime kernel), Sigmoid, ReLU, LeakyReLU, Exp, Log, Sqrt, Rsqrt, Sin, Cos, AddScalar, MulScalar
• MatMul-like / shape: MatMul (uses MatMulTransposeB when the engine implements the optional interface), Transpose, Reshape, HadamardTransform
• Softmax + reductions: Softmax, ReduceSum, ReduceMean, ReduceMax
• LayerNorm-like: parameterized LayerNorm (gamma/beta) that caches xhat/inv-stddev in Forward like the production node -- also exercises the parameter-gradient path.

Non-differentiable points are steered around via OpInfo domains, as PyTorch OpInfo does: positive-only sampling for Log/Sqrt/Rsqrt/Div-denominator/Pow-base; away-from-zero sampling for the ReLU/LeakyReLU kink; continuous sampling avoids ties for ReduceMax.

Found but not registered (with reason): graph.inputNode (unexported identity placeholder, not an op), graph.checkpointNode/CheckpointedSegment (composition wrappers requiring inner nodes, not primitive ops), Gather/ScatterAdd/OneHot (integer-indexed inputs; the checker does not yet support non-differentiable inputs -- noted in code).

Red-proof fixture

TestRedProofWrongJacobianFails: a BadTanh node whose Backward returns 2x the true gradient; the test asserts gradcheck flags it (and fails if the checker ever passes it). TestFreshNodePerEvaluation additionally proves no instance reuse: a fixture that poisons its output on any second Forward of the same instance must still pass.

Test results

• go build ./... clean, gofmt clean
• go vet clean with CI's package set (purego GPU-binding exclusions)
• go test ./... -count=1: 28 packages ok, including testing/gradcheck (27 registry subtests + red proof + fresh-node proof + parameter-gradient + config tests)
• No GPU use anywhere; f64 CPU only.

Intentionally deferred

• T1.2 GPU-vs-CPU parity under arena stress (Spark pod on the GB10)
• T1.3 PyTorch oracle (NGC container, tensor-exchange format)
• T1.6 registering zerfoo layer/timeseries nodes and retiring their ad-hoc finite-difference tests
• Broadcasting elementwise shapes, batched (3D) matmul, and non-differentiable (integer) inputs in the checker.