feat: add host range profiling

[voltjia]

Summary

• Add opt-in host-side range profiling behind INFINI_OPS_ENABLE_HOST_RANGE_PROFILING, disabled by default.
• Instrument binding conversion, generated dispatch, cache/operator invocation, CUDA Add submission, and NVIDIA cuBLASLt GEMM submission.
• Use named HostRangeScope RAII objects at call sites; no instrumentation macro is required.
• Reduce profiling compile-time branches from eight to three deliberate boundaries: implementation selection, real/empty scope type, and the cache hot path.
• Compile profiling-disabled scopes as an inline empty type so optimized builds eliminate them, and keep the profiler header out of installed public headers.
• Extend the existing pytest benchmark flow with --host-range-profile, cold/warm JSONL reports, calibration/control tooling, and focused regression tests.

Motivation

The existing pytest benchmark path measures end-to-end operator latency but cannot attribute CPU-side overhead across Python binding, conversion, dispatch, cache, operator, and backend-submission layers.

This PR adds coarse host attribution without introducing a separate C++ benchmark framework or timing device execution. It is an opt-in diagnostic facility, not a kernel profiler or performance gate.

Related issue: N/A - follows the performance-testing design discussion.

Type of Change

• feat - new feature / new operator / new platform
• fix - bug fix
• perf - performance improvement (no behavioral change)
• refactor - code restructuring without behavior change
• test - adding or fixing tests only
• docs - documentation only
• build / ci - build system or CI configuration
• chore - tooling, formatting, or other non-code changes
• Breaking change

Platforms Affected

• CPU (WITH_CPU)
• NVIDIA (WITH_NVIDIA)
• Iluvatar (WITH_ILUVATAR)
• MetaX (WITH_METAX)
• Cambricon (WITH_CAMBRICON)
• Moore (WITH_MOORE)
• Ascend (WITH_ASCEND)
• PyTorch C++ bindings (WITH_TORCH)
• Build system / CMake / CI
• Python bindings / user-facing API

Smoke Test Result

Final NVIDIA validation ran in accelerator-dev/nvidia:latest on ssh nvidia against commit 072e79680588a535dda9a1f50d0ca40d7436e0fc (tree 2cba932a0c92e06d08f1fca51e40b9ad1bacf74d). Physical GPU 4 was exposed as logical cuda:0.

NVIDIA profiling ON build/install (RelWithDebInfo): passed
NVIDIA profiling OFF build/install (Release): passed
Compile-time guard count: 3
Binary symbol check:
ON libinfiniops.so: HostRangeScope ctor/dtor present
OFF libinfiniops.so: no HostRangeScope symbols
Focused profiling suite:
NVIDIA profiling ON: 43 passed in 29.40s
NVIDIA profiling OFF: 39 passed, 4 skipped in 0.46s
Exact final-SHA reports:
Add profiling ON: 1 passed, 131 deselected in 1.69s; 39 JSONL rows
GEMM profiling ON: 1 passed, 2999 deselected in 1.99s; 39 JSONL rows
backend.submit rows: 4 in each report
Local generator/public-header tests: 20 passed in 0.45s
Ruff 0.15.22 check and format check: passed
clang-format 21.1.8 --dry-run --Werror: passed
git diff --check: passed
Final-SHA GitHub checks:
legacy unit: NVIDIA, Iluvatar, MetaX, Cambricon, Moore, Ascend passed
CI v2 shadow: NVIDIA, Iluvatar, MetaX, Cambricon, Moore, Ascend passed
clang-format, Ruff, documentation build passed; deploy skipped

Earlier CPU validation on pre-RAII commit c4ff40f passed its profiling build/install, focused suite (41 passed), and smoke suite (54 passed, 8 skipped). It was not rerun on the final SHA, so it is retained only as earlier evidence rather than a final-SHA platform claim.

Test Results on Supported Platforms

Platform	Affected	Build / Smoke Result	Full Result / Notes
CPU	Yes	Earlier pre-RAII build/smoke passed (54 passed, 8 skipped)	Final SHA not rerun on CPU; no final-SHA CPU claim
NVIDIA	Yes	Final-SHA legacy and shadow CI passed	Independent profiling ON/OFF builds, focused suites, symbol checks, and exact Add/GEMM reports passed
Iluvatar	Yes	Final-SHA legacy and shadow CI passed	No profiling-enabled device-specific report was run
MetaX	Yes	Final-SHA legacy and shadow CI passed	No profiling-enabled device-specific report was run
Cambricon	Yes	Final-SHA legacy and shadow CI passed	No profiling-enabled device-specific report was run
Moore	Yes	Final-SHA legacy and shadow CI passed	No profiling-enabled device-specific report was run
Ascend	Yes	Final-SHA legacy and shadow CI passed	No profiling-enabled device-specific report was run

Focused pytest output for final SHA

NVIDIA profiling ON:
........................................... [100%]
43 passed in 29.40s
NVIDIA profiling OFF:
ssss....................................... [100%]
39 passed, 4 skipped in 0.46s
Add report:
. [100%]
1 passed, 131 deselected in 1.69s
GEMM report:
. [100%]
1 passed, 2999 deselected in 1.99s

Benchmark / Performance Impact

Final-report measurements used one NVIDIA A100-SXM4-80GB (physical CUDA_VISIBLE_DEVICES=4, logical cuda:0) and the profiling-ON RelWithDebInfo build. Device synchronization occurs only outside collection windows.

• Add: contiguous FP32 (13, 4), implementation 0.
• GEMM: FP32 (4, 48, 64) x (4, 64, 6), cuBLASLt implementation 1.

Warm median	Add	GEMM
end_to_end	30.913 us	39.869 us
binding.body inclusive	18.887 us	25.543 us
dispatch.call inclusive	6.948 us	13.213 us
operator.invoke inclusive	5.486 us	11.538 us
backend.submit inclusive	4.594 us	10.890 us

The two final-SHA JSONL reports contain 39 rows each. add-raii-072e796.jsonl SHA256 is 682b9846e366bb674535beb6a4a5a28b753a005706f8f994201db922973c413f; gemm-raii-072e796.jsonl SHA256 is 2ebfae447276cbc4b1212cba36d50f3295ac03d8fb90f783aabb4ef85e423a10.

An earlier same-process alternating control on the same A100 characterized active-collection overhead at approximately +9.7% for Add and +6.6% to +7.0% for GEMM, with a conservative complete-scope calibration of 134.12 ns. Those values came from a pre-refactor experiment and are observer-effect guidance, not final-tree performance claims.

Complete `add-raii-072e796.jsonl` output

{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.body","metric":"inclusive","count":1,"unit":"ns","mean":775383.0,"median":775383.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.body","metric":"self","count":1,"unit":"ns","mean":2123.0,"median":2123.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"inclusive","count":3,"unit":"ns","mean":10765.666666666666,"median":6305.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"self","count":3,"unit":"ns","mean":10765.666666666666,"median":6305.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"inclusive","count":1,"unit":"ns","mean":7986.0,"median":7986.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"self","count":1,"unit":"ns","mean":7986.0,"median":7986.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"inclusive","count":1,"unit":"ns","mean":732977.0,"median":732977.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"self","count":1,"unit":"ns","mean":1485.0,"median":1485.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.call","metric":"inclusive","count":1,"unit":"ns","mean":731492.0,"median":731492.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.call","metric":"self","count":1,"unit":"ns","mean":229161.0,"median":229161.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.key","metric":"inclusive","count":1,"unit":"ns","mean":3643.0,"median":3643.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.key","metric":"self","count":1,"unit":"ns","mean":3643.0,"median":3643.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"inclusive","count":1,"unit":"ns","mean":551.0,"median":551.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"self","count":1,"unit":"ns","mean":551.0,"median":551.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"inclusive","count":1,"unit":"ns","mean":478299.0,"median":478299.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"self","count":1,"unit":"ns","mean":478299.0,"median":478299.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"inclusive","count":1,"unit":"ns","mean":19838.0,"median":19838.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"self","count":1,"unit":"ns","mean":2157.0,"median":2157.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"inclusive","count":1,"unit":"ns","mean":17681.0,"median":17681.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"self","count":1,"unit":"ns","mean":17681.0,"median":17681.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.body","metric":"inclusive","count":6500,"unit":"ns","mean":19124.904615384614,"median":18887.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.body","metric":"self","count":6500,"unit":"ns","mean":775.2773846153846,"median":674.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"inclusive","count":19500,"unit":"ns","mean":3287.0886153846154,"median":3166.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"self","count":19500,"unit":"ns","mean":3287.0886153846154,"median":3166.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"inclusive","count":6500,"unit":"ns","mean":1399.406923076923,"median":1395.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"self","count":6500,"unit":"ns","mean":1399.406923076923,"median":1395.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"inclusive","count":6500,"unit":"ns","mean":7088.954461538461,"median":6947.5}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"self","count":6500,"unit":"ns","mean":244.76738461538463,"median":229.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.call","metric":"inclusive","count":6500,"unit":"ns","mean":6844.187076923077,"median":6714.5}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.call","metric":"self","count":6500,"unit":"ns","mean":486.9393846153846,"median":449.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.key","metric":"inclusive","count":6500,"unit":"ns","mean":522.2975384615385,"median":512.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.key","metric":"self","count":6500,"unit":"ns","mean":522.2975384615385,"median":512.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"inclusive","count":6500,"unit":"ns","mean":256.622,"median":257.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"self","count":6500,"unit":"ns","mean":256.622,"median":257.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"inclusive","count":6500,"unit":"ns","mean":5578.328153846154,"median":5486.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"self","count":6500,"unit":"ns","mean":917.5878461538462,"median":886.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"inclusive","count":6500,"unit":"ns","mean":4660.740307692307,"median":4594.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"self","count":6500,"unit":"ns","mean":4660.740307692307,"median":4594.0}
{"nodeid":"tests/test_add.py::test_add[cuda-0-input_shape0-other_shape0-out_shape0-None-None-None-None-dtype0-1e-07-1e-07]","operator":"add","backend":"nvidia","phase":"warm","range":"end_to_end","metric":"inclusive","count":6500,"unit":"ns","mean":30906.074895308568,"median":30913.334339857105}

Complete `gemm-raii-072e796.jsonl` output

{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.body","metric":"inclusive","count":1,"unit":"ns","mean":113921.0,"median":113921.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.body","metric":"self","count":1,"unit":"ns","mean":1824.0,"median":1824.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"inclusive","count":3,"unit":"ns","mean":12624.0,"median":7883.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.tensor_conversion","metric":"self","count":3,"unit":"ns","mean":12624.0,"median":7883.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"inclusive","count":1,"unit":"ns","mean":8177.0,"median":8177.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"binding.device_conversion","metric":"self","count":1,"unit":"ns","mean":8177.0,"median":8177.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"inclusive","count":1,"unit":"ns","mean":66048.0,"median":66048.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"dispatch.call","metric":"self","count":1,"unit":"ns","mean":845.0,"median":845.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.call","metric":"inclusive","count":1,"unit":"ns","mean":65203.0,"median":65203.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.call","metric":"self","count":1,"unit":"ns","mean":3445.0,"median":3445.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.key","metric":"inclusive","count":1,"unit":"ns","mean":3257.0,"median":3257.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.key","metric":"self","count":1,"unit":"ns","mean":3257.0,"median":3257.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"inclusive","count":1,"unit":"ns","mean":268.0,"median":268.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.lookup","metric":"self","count":1,"unit":"ns","mean":268.0,"median":268.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"inclusive","count":1,"unit":"ns","mean":6145.0,"median":6145.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"cache.construct","metric":"self","count":1,"unit":"ns","mean":6145.0,"median":6145.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"inclusive","count":1,"unit":"ns","mean":52088.0,"median":52088.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"operator.invoke","metric":"self","count":1,"unit":"ns","mean":799.0,"median":799.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"inclusive","count":1,"unit":"ns","mean":51289.0,"median":51289.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"cold","range":"backend.submit","metric":"self","count":1,"unit":"ns","mean":51289.0,"median":51289.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.body","metric":"inclusive","count":5000,"unit":"ns","mean":27977.3664,"median":25542.5}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.body","metric":"self","count":5000,"unit":"ns","mean":896.288,"median":734.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"inclusive","count":15000,"unit":"ns","mean":3718.7997333333333,"median":3262.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.tensor_conversion","metric":"self","count":15000,"unit":"ns","mean":3718.7997333333333,"median":3262.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"inclusive","count":5000,"unit":"ns","mean":1573.5644,"median":1426.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"binding.device_conversion","metric":"self","count":5000,"unit":"ns","mean":1573.5644,"median":1426.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"inclusive","count":5000,"unit":"ns","mean":14351.1148,"median":13213.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"dispatch.call","metric":"self","count":5000,"unit":"ns","mean":245.727,"median":209.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.call","metric":"inclusive","count":5000,"unit":"ns","mean":14105.3878,"median":13003.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.call","metric":"self","count":5000,"unit":"ns","mean":545.413,"median":463.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.key","metric":"inclusive","count":5000,"unit":"ns","mean":844.6874,"median":771.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.key","metric":"self","count":5000,"unit":"ns","mean":844.6874,"median":771.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"inclusive","count":5000,"unit":"ns","mean":253.9402,"median":226.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"cache.lookup","metric":"self","count":5000,"unit":"ns","mean":253.9402,"median":226.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"inclusive","count":5000,"unit":"ns","mean":12461.3472,"median":11538.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"operator.invoke","metric":"self","count":5000,"unit":"ns","mean":709.7614,"median":639.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"inclusive","count":5000,"unit":"ns","mean":11751.5858,"median":10890.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"backend.submit","metric":"self","count":5000,"unit":"ns","mean":11751.5858,"median":10890.0}
{"nodeid":"tests/test_gemm.py::test_gemm[cuda-1-dtype0-0.001-0.001-False-False-0-1-a_shape4-b_shape4-c_shape4-None-None-None]","operator":"gemm","backend":"nvidia","phase":"warm","range":"end_to_end","metric":"inclusive","count":5000,"unit":"ns","mean":40760.25132089853,"median":39868.587628006935}

Notes for Reviewers

• backend.submit measures host API work through API return; it does not measure device execution or kernel duration. CUDA APIs may still block on queue/device progress.
• Synchronization occurs only before and after cold/warm collection windows.
• end_to_end and replayed C++ ranges use separate populations and must not be subtracted from one another.
• Profiling is disabled by default; both compiled-ON and compiled-OFF builds were tested.
• Call sites use named RAII objects rather than preprocessor macros.
• The eight original source-level conditions are reduced to three: host_range_profiler.cc selects the implementation, host_range_profiler.h selects the real or empty scope, and operator.h preserves the exact profiling-OFF cache hot path.
• In optimized OFF builds the empty scope is eliminated; the final OFF library contains no HostRangeScope symbols.
• host_range_profiler.h is internal and is excluded from both installed headers and the generated public operator.h, avoiding an ON/OFF class-definition contract for downstream consumers.
• cuBLASLt instrumentation is limited to one include and one function-entry scope, matching the Add backend path without restructuring the function body.
• pytest-xdist is intentionally rejected when host-range output is enabled.
• Backend-submission instrumentation currently covers the shared CUDA Add path and NVIDIA cuBLASLt GEMM. Other operators still receive common binding, dispatch, cache, and invocation ranges.
• Raw reports are included above for review but intentionally are not committed as machine-specific repository artifacts.
• All final-SHA legacy and shadow platform checks passed; no CI rerun was requested or triggered manually.
• Process-only design and planning files under docs/superpowers/ were removed from the final branch.