Skip to content

feat(linked): add MetaX vLLM silu_and_mul backend - #873

Merged
voltjia merged 11 commits into
masterfrom
feat/linked-metax-silu-and-mul
Aug 7, 2026
Merged

feat(linked): add MetaX vLLM silu_and_mul backend#873
voltjia merged 11 commits into
masterfrom
feat/linked-metax-silu-and-mul

Conversation

@voltjia

@voltjiavoltjia commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add src/linked as a peer implementation backend for operators recovered from installed shared libraries.
  • Support multiple provider implementations for one (operator, device) through provider-named YAML/C++ files and independent implementation slots.
  • Organize linked implementations under src/linked/<transport>/<device> and add MetaX vLLM silu_and_mul at implementation slot 16.

Motivation

Some accelerator vendors distribute optimized vLLM operators only as shared objects. InfiniOps needs a narrow, auditable integration path for those implementations without vendoring generated kernels, adding a submodule, or treating them as native source implementations.

The public API remains InfiniOps silu_and_mul(input, out), with packed-input semantics following vLLM SiluAndMul. The installed MetaX provider ABI is verified independently by nm and readelf as:

silu_and_mul(at::Tensor&, at::Tensor&)

N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • Hygon (WITH_HYGON)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Environment: ssh metax, image cr.metax-tech.com/public-ai-release/maca/vllm-metax:0.10.2-maca.ai3.2.1.7-torch2.6-py310-ubuntu22.04-amd64 (sha256:6e519687a9e4...), exact commit 20ceb1cc50f575449ba258a746f736e3bd6327f9.

INFINI_OPS_OPS=silu_and_mul, WITH_METAX=ON, WITH_LINKED=ON, WITH_TORCH=OFF
python -m pytest tests -m smoke --devices metax -q
46 passed, 40 skipped, 4269 deselected

Test Results on Supported Platforms

PlatformAffectedBuild / Smoke ResultFocused Result / Notes
MetaXYesOperator-pruned linked build and smoke passedtests/test_silu_and_mul.py: 50 passed
Other platformsNoN/AAdded by separate stacked PRs
Additional validation
tests/test_resolve_linked_ops.py + tests/test_generate_wrappers.py
38 passed
libinfiniops.so NEEDED: vllm/_C.abi3.so, torch, c10, and MetaX runtime libraries
RUNPATH: $ORIGIN:<torch/lib>:<vllm>:<MetaX runtime lib>

Benchmark / Performance Impact

N/A - this PR establishes an integration path and makes no performance claim.

Notes for Reviewers

  • Linked providers live under src/linked/<transport>/<device>. Platform-level <library>.yaml files discover DSOs; operator-level provider files use one stem, for example vllm.yaml, vllm.h, and vllm.cc.
  • Platform library YAML contains python_distribution_package and library_glob for DSO discovery.
  • Operator binding YAML identifies its DSO with library: vllm; multiple implementations may reference the same DSO.
  • Linked implementations use unique slots from 16; kMaxImplementations is 32.
  • TorchSiluAndMul<Backend> owns tensor wrapping, contiguous staging, and output copy-back. Each provider Backend inherits C10<device> for device identity and stream handling, then adds only its exact Call ABI.
  • The shared Torch adapter declaration and implementation live together in src/linked/torch/ops/silu_and_mul.h. Provider .cc remains the ABI and explicit-instantiation boundary; provider .h uses extern template so generated translation units do not instantiate it again.
  • C10<device> converts the caller's external stream to the vendor Torch stream type and exposes the matching stream guard; it does not replace the caller-selected stream.
  • The provider DSO is linked but not bundled. The resolver rejects ambiguous libraries, missing symbols, and the same global C++ symbol exported by multiple loaded DSOs.
  • Loading multiple provider DSOs that export the same global C++ symbol is unsupported because of ELF symbol interposition.

@voltjiavoltjia mentioned this pull request Aug 4, 2026
20 tasks
@voltjia
voltjia marked this pull request as ready for review August 6, 2026 08:08
@voltjia
voltjia requested a review from a teamAugust 6, 2026 08:08
@voltjiavoltjia changed the title feat(linked): add MetaX vLLM silu_and_mul backendfeat(linked): add MetaX vLLM silu_and_mul backendAug 6, 2026
@voltjiavoltjia mentioned this pull request Aug 6, 2026
19 tasks
@voltjia
voltjia merged commit d0a2bd4 into masterAug 7, 2026
16 of 20 checks passed
@voltjia
voltjia deleted the feat/linked-metax-silu-and-mul branch August 7, 2026 10:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@voltjia@whjthu@Ziminli