Skip to content

feat(moore): link vLLM silu_and_mul - #874

Merged
voltjia merged 3 commits into
feat/linked-metax-silu-and-mulfrom
feat/linked-moore-silu-and-mul
Aug 7, 2026
Merged

feat(moore): link vLLM silu_and_mul#874
voltjia merged 3 commits into
feat/linked-metax-silu-and-mulfrom
feat/linked-moore-silu-and-mul

Conversation

@voltjia

@voltjiavoltjia commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add the Moore vLLM mapping for silu_and_mul(at::Tensor&, at::Tensor&) as implementation slot 16.
  • Name the implementation vllm.yaml, vllm.h, and vllm.cc, reusing the shared linked Torch silu_and_mul template.
  • Bridge the caller-provided MUSA stream through C10<Device::Type::kMoore>.

Motivation

Moore vLLM exposes an optimized silu_and_mul implementation through a C++ ABI. InfiniOps links it to the public silu_and_mul(input, out) API. C10<Device::Type::kMoore> owns device and stream integration, while VllmSiluAndMul owns the exact external ABI call.

Semantics follow vLLM SiluAndMul. The pinned Moore image exports:

silu_and_mul(at::Tensor&, at::Tensor&)

Depends on #873. N/A - no linked issue.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • Hygon (WITH_HYGON)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

Manual runtime evidence at 44413c861c55d825da4b46d3a4d716f8391b96b2: ssh huangjiacheng@172.22.162.98, image zx_vllm_musa_436:vllm017-cli-ok-mate-disabled (sha256:0431d022137d...), Torch 2.7.1, vLLM 0.17.1.dev0+gb31e9326a.d20260615.empty.

INFINI_OPS_OPS=silu_and_mul, WITH_MOORE=ON, WITH_LINKED=ON, WITH_TORCH=OFF
python3 -m pytest tests/test_silu_and_mul.py -m smoke --devices moore -q
4 passed, 46 deselected

At current head d70f191b, both Moore main CI and Moore shadow CI pass.

Test Results on Supported Platforms

PlatformAffectedBuild / Smoke ResultFocused Result / Notes
MooreYesManual operator-pruned smoke passed; current-head main and shadow CI passtests/test_silu_and_mul.py: 50 passed at 44413c86
MetaXNoN/AProvided by dependency #873
Other platformsNoN/ANot affected
Additional validation
tests/test_resolve_linked_ops.py + tests/test_generate_wrappers.py
39 passed
NEEDED: vLLM _C, libtorch_cpu.so, libc10.so, libmusart.so.4, libmublas.so.1
RUNPATH: $ORIGIN:<torch/lib>:<vllm>

Benchmark / Performance Impact

N/A - this PR changes integration behavior and makes no performance claim.

Notes for Reviewers

  • Moore implementation files use the vllm.* stem. Platform library YAML contains the Python package and DSO glob.
  • TorchSiluAndMul<Backend> provides tensor conversion, staging, and copy-back. VllmSiluAndMul::Call contains the exact external ABI call.
  • C10<Device::Type::kMoore> uses c10::musa::getStreamFromExternal and c10::StreamGuard. The guard applies the caller-selected device and stream.
  • The vendor compatibility header <c10/musa/MUSAMacros.h> remains required because that image's MUSAStream.h does not define C10_MUSA_API itself.
  • The non-default-stream test is included in the 50 passed result.

@voltjia
voltjiaforce-pushed the feat/linked-moore-silu-and-mul branch from e75f5f7 to 44413c8CompareAugust 5, 2026 11:45
@voltjia
voltjiaforce-pushed the feat/linked-moore-silu-and-mul branch from 44413c8 to d70f191CompareAugust 6, 2026 08:24
@voltjia
voltjia marked this pull request as ready for review August 6, 2026 09:07
@voltjia
voltjia requested a review from a teamAugust 6, 2026 09:07
@voltjiavoltjia changed the title feat(moore): link vLLM silu_and_mulfeat(moore): link vLLM silu_and_mulAug 6, 2026
@voltjiavoltjia mentioned this pull request Aug 6, 2026
19 tasks
@voltjia
voltjia merged commit 9bcf944 into masterAug 7, 2026
16 of 20 checks passed
@voltjia
voltjia deleted the feat/linked-moore-silu-and-mul branch August 7, 2026 10:58
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@voltjia@whjthu@Ziminli