Skip to content

fix: align graph runtime API with runtime namespace - #13

Merged
voltjia merged 6 commits into
feat/graph-runtime-apifrom
fix/graph-runtime-api-runtime-namespace
Jul 3, 2026
Merged

fix: align graph runtime API with runtime namespace#13
voltjia merged 6 commits into
feat/graph-runtime-apifrom
fix/graph-runtime-api-runtime-namespace

Conversation

@voltjia

@voltjiavoltjia commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Move generated C++ runtime wrappers under infini::rt::runtime so that API names matching CUDA Runtime API stay in that nested namespace.
  • Keep the outer infini::rt namespace focused on runtime-device selection APIs.
  • Adapt graph C++ tests and C API implementation to the runtime namespace; C API graph handles remain opaque wrappers around backend/raw handles.
  • Generate graph runtime declarations only when an enabled backend implements graph APIs, so CPU-only builds do not expose unsupported graph symbols.
  • Refresh docs and install-consumer tests for the new public API layout.

Motivation

This helper PR lets feat/graph-runtime-api absorb the current master runtime API layout before the graph-runtime PR is rebased. It preserves the project decision that CUDA-aligned Runtime API names live under infini::rt::runtime, while InfiniRT-specific selection APIs live directly under infini::rt.

No issue is closed directly by this PR.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

# ssh nvidia, image accelerator-dev/nvidia:latest
cmake -B build -DWITH_CPU=ON -DWITH_NVIDIA=ON \
-DINFINI_RT_BUILD_TESTING=ON \
-DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure
100% tests passed, 0 tests failed out of 9
# CPU-only generated API check, same image
cmake -B build-cpu -DWITH_CPU=ON \
-DINFINI_RT_BUILD_TESTING=ON \
-DCMAKE_BUILD_TYPE=Release
cmake --build build-cpu -j8
ctest --test-dir build-cpu --output-on-failure
100% tests passed, 0 tests failed out of 6
clang-format 21 dry-run: passed
ruff format --check scripts/generate_public_headers.py: passed
# Integration check with simulated #13 merged into #9 and rebased onto master,
# InfiniCore#1351, InfiniLM commit 36e2207, model Qwen3-0.6B.
python3 examples/test_infer.py --device nvidia --enable-graph \
--model=/data-aisoft/mechdancer/models/Qwen3-0.6B/
Standalone InfiniRT graph bridge loaded: /workspace/verify/infini-rt-prefix/lib/libinfinirt.so
Using standalone InfiniRT graph bridge for graph capture and replay.
[codex-verify] InfiniRT C API infiniRtStreamBeginCapture: 1
[codex-verify] InfiniRT C API infiniRtStreamEndCapture: 1
[codex-verify] InfiniRT C API infiniRtGraphInstantiate: 1
[codex-verify] InfiniRT C API infiniRtGraphLaunch: 511
Inference completed, total_time: 1448.2 ms

Test Results on Supported Platforms

PlatformAffectedBuild / Smoke ResultFull Result / Notes
NVIDIAYessmoke passedFull graph integration passed with InfiniCore#1351 and InfiniLM as above.
IluvatarYesnot rerun in this helper PRAPI/generation touched; backend validation to run on the rebased #9 branch.
MetaXYesnot rerun in this helper PRAPI/generation touched; backend validation to run on the rebased #9 branch.
CambriconYesnot rerun in this helper PRAPI/generation touched; backend validation to run on the rebased #9 branch.
MooreYesnot rerun in this helper PRAPI/generation touched; backend validation to run on the rebased #9 branch.
AscendYesnot rerun in this helper PRAPI/generation touched; backend validation to run on the rebased #9 branch.
Full `pytest` output (optional)
N/A - no pytest suite in this PR.

Benchmark / Performance Impact

N/A - this PR changes API placement, wrapper generation, tests, and docs; it is not performance-sensitive.

Notes for Reviewers

  • This is a helper PR targeting feat/graph-runtime-api, not master directly.
  • It intentionally keeps CUDA-aligned names under infini::rt::runtime; outer infini::rt remains available for InfiniRT-specific device-type selection APIs.
  • Current InfiniLM main has newer InfiniCore RoPE/mrope API expectations than InfiniCore#1351, so the graph integration smoke used InfiniLM commit 36e2207ae6eaa4296b4564895648d50aaeef5453, which is compatible with #1351.

voltjiaand others added 2 commits July 3, 2026 09:18
* Align runtime API with generated wrappers
* Add default runtime dispatch specialization
* Refactor runtime dispatch namespace
* Use Abseil status for runtime device API
* Revert "Use Abseil status for runtime device API"
This reverts commit a26ddff.
* Address runtime dispatch review feedback
* Keep runtime API list in generator
* Add TensorView constructor guard test
* Align runtime memcpy kind constants with CUDA API
* Use CUDA-style runtime memcpy constants
* Use CUDA-style runtime memcpy constants
* Move TensorView tests back into core test
* Remove standalone TensorView test target
* Remove standalone TensorView test file
* Use fully qualified runtime API names in README
* style: format runtime dispatch test
@voltjia
voltjiaforce-pushed the fix/graph-runtime-api-runtime-namespace branch from 87fb536 to 943be4bCompareJuly 3, 2026 02:06
spike-zhuand others added 4 commits July 3, 2026 11:15
Co-authored-by: Jiacheng Huang <huangjiacheng0709@outlook.com>
# Conflicts:
#	scripts/generate_public_headers.py
#	src/native/cuda/iluvatar/runtime_.h
#	src/native/cuda/metax/runtime_.h
#	src/native/cuda/moore/runtime_.h
#	src/native/cuda/nvidia/runtime_.h
#	tests/CMakeLists.txt
@voltjia
voltjia merged commit 7036180 into feat/graph-runtime-apiJul 3, 2026
4 checks passed
@voltjia
voltjia deleted the fix/graph-runtime-api-runtime-namespace branch July 3, 2026 08:01
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

@voltjia@spike-zhu