Skip to content

feat: add graph runtime API - #9

Merged
voltjia merged 8 commits into
masterfrom
feat/graph-runtime-api
Jul 8, 2026
Merged

feat: add graph runtime API#9
voltjia merged 8 commits into
masterfrom
feat/graph-runtime-api

Conversation

@gongchensu

@gongchensugongchensu commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds generated C++ runtime declarations and dispatch support for stream capture and graph lifecycle APIs in scripts/generate_public_headers.py.
  • Adds native graph API coverage for CUDA-like backends in src/native/cuda/nvidia/runtime_.h, src/native/cuda/iluvatar/runtime_.h, and src/native/cuda/ascend/runtime_.h.
  • Adds an adaptive graph runtime test in tests/test_native_graph.cc, wired through tests/CMakeLists.txt.

Motivation

InfiniRT should expose CUDA Runtime API-aligned graph capture and graph launch APIs through the generated C++ runtime API, without introducing a C API. This is needed by the standalone InfiniRT graph bridge work in InfiniCore.

Related: InfiniTensor/InfiniCore#1351

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

# ssh nvidia, image: infinirt-verify:nvidia-latest
clang-format --dry-run --Werror include/infini/rt.h tests/test_native_graph.cc
cmake -B build \
-DWITH_CPU=ON \
-DWITH_NVIDIA=ON \
-DINFINI_RT_BUILD_TESTING=ON \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/workspace/infini-rt-prefix
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure
100% tests passed, 0 tests failed out of 8
Passed tests include test_nvidia_graph and test_runtime_dispatch.

Test Results on Supported Platforms

PlatformAffectedBuild / Smoke ResultFull Result / Notes
NVIDIAYessmoke passedctest passed: 8/8
IluvatarYesnot runNo Iluvatar validation in this round
MetaXNonot runN/A
CambriconNonot runN/A
MooreNonot runN/A
AscendYesnot runNo Ascend validation in this round
Full `pytest` output (optional)
N/A

Benchmark / Performance Impact

N/A. This PR adds runtime API surface and tests; it does not tune kernel performance.

Notes for Reviewers

  • This PR intentionally keeps InfiniRT as C++ API only. No C API is added.
  • The public runtime graph API is generated under infini::rt::runtime.

@gongchensugongchensu self-assigned this Jun 24, 2026
@gongchensu
gongchensu requested a review from voltjiaJune 24, 2026 04:24
Comment threadtests/CMakeLists.txt Outdated
Comment threadtests/test_nvidia_graph.cc Outdated
Comment threadsrc/graph.h Outdated
Comment threadsrc/native/cuda/runtime_.h Outdated
Comment threadsrc/native/cuda/runtime_.h Outdated
Comment threadsrc/native/cpu/runtime_.h Outdated
Comment threadsrc/native/ascend/runtime_.h
Comment threadsrc/native/ascend/runtime_.h Outdated
Comment threadsrc/native/cambricon/runtime_.h Outdated
Comment threadsrc/native/cambricon/runtime_.h Outdated
@voltjia
voltjia requested a review from ZiminliJune 24, 2026 08:49
Comment threadsrc/native/cuda/runtime_.h
@gongchensu
gongchensuforce-pushed the feat/graph-runtime-api branch 3 times, most recently from aae1676 to a4bac7fCompareJune 25, 2026 09:13
Comment threadsrc/graph.h Outdated
Comment threadsrc/graph.h Outdated
Comment threadsrc/graph.h Outdated
@gongchensu
gongchensuforce-pushed the feat/graph-runtime-api branch from a4bac7f to b9c8badCompareJune 30, 2026 06:31
@voltjia
voltjiaforce-pushed the feat/graph-runtime-api branch 2 times, most recently from 81df7b1 to 9f82a06CompareJuly 3, 2026 08:54
gongchensuand others added 5 commits July 7, 2026 06:43
* feat!: align runtime API and add runtime dispatch (#11)
* Align runtime API with generated wrappers
* Add default runtime dispatch specialization
* Refactor runtime dispatch namespace
* Use Abseil status for runtime device API
* Revert "Use Abseil status for runtime device API"
This reverts commit a26ddff.
* Address runtime dispatch review feedback
* Keep runtime API list in generator
* Add TensorView constructor guard test
* Align runtime memcpy kind constants with CUDA API
* Use CUDA-style runtime memcpy constants
* Use CUDA-style runtime memcpy constants
* Move TensorView tests back into core test
* Remove standalone TensorView test target
* Remove standalone TensorView test file
* Use fully qualified runtime API names in README
* style: format runtime dispatch test
* feat: refactor InfiniCore CPU runtime to InfiniRT (#8)
Co-authored-by: Jiacheng Huang <huangjiacheng0709@outlook.com>
* feat: add platform-adaptive runtime tests (#15)
* feat: add runtime backend API foundation (#14)
---------
Co-authored-by: spike-zhu <74974704+spike-zhu@users.noreply.github.com>
@gongchensu
gongchensuforce-pushed the feat/graph-runtime-api branch from 2fc8a16 to 3daf6cbCompareJuly 7, 2026 06:43
Map the Ascend graph lifecycle to aclmdlRI capture and replay symbols loaded from AscendCL at runtime.
Expose Ascend graph capture and replay through the generated C++ runtime API and link dl only for WITH_ASCEND builds.
@gongchensu
gongchensuforce-pushed the feat/graph-runtime-api branch from b8de364 to a0e5246CompareJuly 7, 2026 07:09
Comment threadsrc/native/cuda/nvidia/runtime_.h Outdated
Comment threadsrc/native/cuda/nvidia/runtime_.h Outdated
Comment threadsrc/native/cuda/moore/runtime_.h
Comment threadsrc/native/cuda/metax/runtime_.h
Comment threadsrc/native/cuda/iluvatar/runtime_.h Outdated
Comment threadsrc/native/ascend/runtime_.h Outdated
Comment threadsrc/native/ascend/runtime_.h Outdated
Comment threadsrc/native/ascend/runtime_.h Outdated
Comment threadsrc/native/ascend/runtime_.h
Move graph runtime validation into the common Runtime contract and keep StreamCreate aligned with CUDA semantics.
Hide Ascend RI symbol probing helpers, use the Runtime Error alias consistently, and drop unverified Moore/Metax graph placeholders.
@gongchensu
gongchensuforce-pushed the feat/graph-runtime-api branch from a0e5246 to 0eb12cbCompareJuly 8, 2026 03:16
@voltjia
voltjia marked this pull request as ready for review July 8, 2026 03:36
@voltjia
voltjia requested review from Ziminli and voltjiaJuly 8, 2026 03:36
@voltjiavoltjia changed the title feat: add graph runtime apifeat: add graph runtime APIJul 8, 2026
Route the Ascend runtime MemsetAsync API to aclrtMemsetAsync so Core can clear device buffers through the InfiniRT C++ runtime path.
@voltjia
voltjia merged commit 25eb81c into masterJul 8, 2026
6 checks passed
@voltjia
voltjia deleted the feat/graph-runtime-api branch July 8, 2026 11:57
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@gongchensu@voltjia@Ziminli