Bench: broaden the JSC matrix beyond specialized kernels

[chrisbbreuer]

Parent: #50
Related: #47, #48, #52

Problem

The accepted comparison is reproducible and symmetric, but its four kernels deliberately exercise paths zig-js has already specialized. Winning those rows does not prove broader call/object/allocation throughput, and a geometric mean over only those kernels is too easy to over-read.

Scope

• add directly comparable workloads for ordinary direct calls, method calls with this, closure creation/capture, and arguments;
• keep recursion as a separate observable workload and retain arithmetic/property/array baselines;
• use the exact same host-API-free source and invocation bytes in both engines;
• return exact deterministic checksums across direct, independent steady/cold, and zig-js shared modes;
• calibrate equal job counts so every full-run engine/mode row clears the publication timing floor without making development runs impractical;
• report per-workload results before aggregates and describe what each kernel does;
• prohibit source text, function-name, or benchmark-name recognition in optimizations.

Acceptance criteria

• Add focused call, method, closure, and arguments workloads with semantic checksums.
• Extend harness validation so missing/extra rows and checksum drift fail deterministically.
• Capture initial symmetric evidence and link every losing row to an optimization issue.
• Optimize semantics-preserving general engine paths until zig-js beats JSC in every direct and independent-context row.
• Keep no-GIL shared scaling above 1.0x at every reported lane count.
• Publish one clean full report/TSV and generated README summary only after the expanded matrix passes all gates.
• Batch the expensive full matrix; use focused runner A/B and quick structural validation during development.

[chrisbbreuer]

First expanded row is on main in dcbf4684 (bench: add direct call comparison workload (#60)). It adds an ordinary global direct-call loop to the exact shared source and matrix, calibrates the full row at 400 jobs (4,000,000 calls), and makes aggregate prose derive the workload count instead of hard-coding four.

Initial focused evidence (3 samples, one job / 10,000 calls):

• zig-js median: ~1.344 ms
• JSC median: ~0.217 ms
• zig-js is ~6.19x slower
• exact checksum: 994854 in both engines

Shared one/eight-lane probes and JSC independent eight-lane produced the same lane-aware checksum (7958860 at eight lanes); shared effective scaling was ~1.24x even at this deliberately tiny thread-overhead-dominated probe. Harness contract remains 7/7. This honest losing row is now the next #48 optimization target; no full matrix will be published until it wins.

[chrisbbreuer]

Direct-call row is now calibrated, optimized, and pushed to main in 4d91efcc + 6d335cdd.

Fairness detail: benchmarkDirectCalls copies the ordinary global function into one lane-local variable before the timed call loop. Both engines still execute the exact same ordinary JavaScript calls, while the shared-realm result cannot be distorted by a race on a mutable benchmark-global binding.

Five-sample medians on the reference M3 Pro at 500 jobs/lane:

• single: zig-js 63.979 ms, JSC 97.644 ms (zig-js 1.53x throughput)
• independent steady 1 lane: 64.968 ms vs 99.079 ms (1.52x)
• independent steady 8 lanes: 83.990 ms vs 159.983 ms (1.90x)
• independent cold 1 lane: 64.987 ms vs 99.981 ms (1.54x)
• independent cold 8 lanes: 89.921 ms vs 162.294 ms (1.80x)
• zig-js shared realm: 63.942 ms at 1 lane, 85.428 ms at 8 lanes = 5.99x scaling

Checksums matched exactly in every directly comparable row (497551750 at one lane; 3980428000 at eight lanes). Every full median clears the 50 ms publication floor.

The five-workload quick matrix also passed. Leaving this issue open until method calls, closures, and arguments broaden the matrix and the final full saved report/README are regenerated once.

[chrisbbreuer]

Method-call coverage is now on main in 38b7e923 (bench: add method call comparison workload (#60)).

The exact shared source creates one lane-local receiver with a bias data property and step method. Every inner iteration executes receiver.step(value, i), and the callee reads this.bias; this prevents treating the row as a decorative direct call while avoiding a shared mutable benchmark receiver.

Initial focused evidence, 3 samples at 1 job / 10,000 method calls:

• zig-js single median: 1.114 ms
• JSC single median: 0.293 ms
• JSC currently has ~3.80x zig-js throughput
• exact one-lane checksum: 4851 in both engines
• exact eight-lane checksum: 318808 in zig-js independent/shared and JSC independent modes
• zig-js shared median: 1.490 ms at 1 lane, 2.671 ms at 8 lanes, or 4.46x effective scaling

The calibrated full row is 500 equal jobs/lane; JSC projects above the 50 ms publication floor. This losing row is linked to #48 for a general guarded method dispatch optimization. No full report/README regeneration yet.

[chrisbbreuer]

The method row now wins every fair JSC comparison after 03061dd1 (details and medians in #48). The complete six-workload quick matrix passes, the full 500-job row clears the 50 ms floor everywhere, and zig-js shared scaling is 4.63x at eight lanes.

Remaining matrix breadth before final publication: closure creation/capture and arguments. The full report/TSV/README is intentionally still deferred until those rows are present, optimized, and gated together.

[chrisbbreuer]

Closure creation/capture coverage is on main in 844efa1f (bench: add closure call comparison workload (#60)).

Each inner iteration creates a fresh anonymous function, captures the job's mutable seed by reference, calls the closure once, and feeds the result back into the captured slot. The identity does not escape the iteration, but the capture and allocation are real; this gives both engines the same opportunity for escape analysis/scalar replacement. Each job performs 10,000 closure creations and calls.

Initial 3-sample focused evidence at one job:

• zig-js single median: 24.471 ms
• JSC single median: 0.701 ms
• JSC currently has ~34.9x zig-js throughput
• exact checksum: 994854 in both engines

Eight lanes produce exact checksum 7958860. zig-js independent steady median is 55.962 ms versus JSC 0.824 ms. Shared zig-js is 7.957 ms at one lane and 243.642 ms at eight lanes, only 0.26x effective scaling, exposing shared allocation/capture contention rather than a checksum race.

The calibrated full row is 200 jobs/lane (JSC projects safely above the 50 ms floor). Direct/independent optimization is tracked in #48; shared allocation scaling is also linked to #49. Full publication remains deferred.

[chrisbbreuer]

The closure workload is now optimized on main by 3e92b41. Production-size validation (200 jobs/lane, 3-sample medians) has zig-js ahead of JSC in all five directly comparable rows: 2.50x single, 2.17x steady/1, 2.52x steady/8, 2.41x cold/1, and 2.94x cold/8. The complete quick matrix also passed with stable cross-engine checksums.

This completes the implementation/optimization checkpoint for direct, method, and closure call shapes. I am leaving #60 open for the arguments workload and one final clean seven-sample report/TSV/README refresh, so the expensive publication run happens once.

[chrisbbreuer]

The final breadth row landed on main in 9b33f9b (bench: add arguments object comparison workload (#60)). It calls a two-parameter function that deliberately reads arguments[0] and arguments[1], so this is real arguments-object materialization/indexing rather than an unused syntax marker. Default full-run size is 200 jobs/lane; the harness remains structurally validated (7/7 tests).

Honest 1-job baseline (3-sample medians, exact checksum 994854): zig-js 34.908 ms vs JSC 0.536 ms, about 65.1x behind. Shared 8-lane median is 586.048 ms for checksum 7958860. This is the expected optimization target; no favorable result is being published before the engine work.

[chrisbbreuer]

All four required call-shape breadth rows are now implemented and optimized on main. The arguments row was completed by bac1fa5 and is ahead of JSC in all five production-size fair comparisons by 3.05x–3.90x in the current 3-sample validation. The complete eight-workload quick matrix passes with stable cross-engine checksums.

The remaining #60 work is now publication/closure only: one clean seven-sample full matrix with TSV + Markdown, README/docs refresh from that exact commit, full suite/TSan batch, and final issue reconciliation.