Skip to content

refactor(ops): migrate InfiniLM adapters to canonical InfiniOps APIs - #1480

Merged
voltjia merged 6 commits into
mainfrom
refactor/migrate-infiniops-canonical-adapters
Aug 8, 2026
Merged

refactor(ops): migrate InfiniLM adapters to canonical InfiniOps APIs#1480
voltjia merged 6 commits into
mainfrom
refactor/migrate-infiniops-canonical-adapters

Conversation

@voltjia

@voltjiavoltjia commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

What

  • Replace the remaining covered InfiniLM-suffixed InfiniOps adapter calls with canonical InfiniOps APIs for activations, paged caching, top-k softmax, softmax, rearrange, and conv2d.
  • Advance the InfiniOps submodule from 1b9c3765 to the common validated pin 21b07ebc.
  • Preserve all public InfiniCore operator signatures and keep each migration as a separate commit.

Alignment

InfiniCore adapterCanonical InfiniOps callOpen-source alignment
geluGelu(input, approximate="none", out)PyTorch torch.nn.functional.gelu, InfiniOps API
gelutanhGelu(input, approximate="tanh", out)PyTorch torch.nn.functional.gelu, InfiniOps #889
reluRelu(input, out)PyTorch torch.nn.functional.relu, InfiniOps API
sigmoidSigmoid(input, out)PyTorch torch.sigmoid, InfiniOps API
silu_and_mulSiluAndMul(input, out)vLLM SiluAndMul, InfiniOps API
paged_cachingReshapeAndCacheFlash(key, value, slot_mapping, k_scale, v_scale, "auto", key_cache, value_cache)vLLM reshape_and_cache_flash, InfiniOps #883
topksoftmaxTopkSoftmax(gating_output, bias, is_padding, renormalize, topk_weights, topk_indices, token_expert_indices)vLLM topk_softmax, InfiniOps API
softmaxSoftmax(input, dim, dtype=nullopt, out)PyTorch torch.nn.functional.softmax, InfiniOps #890
rearrangeCopy(input, non_blocking=false, out)PyTorch Tensor.copy_, InfiniOps #872, provider adaptation #897
conv2dConvolution(input, weight, bias, stride, padding, dilation, false, output_padding=0, groups=1, out)PyTorch torch.nn.functional.conv2d, ATen convolution schema, InfiniOps #882

InfiniOps uses its C++ input/attribute/output ordering. Optional top-k softmax inputs are null because the current InfiniCore API does not expose them; the required auxiliary output is plan-owned internal storage. Paged caching uses metadata-only HND-to-NHD cache views and a persistent scalar scale tensor ignored by the current auto path.

Scope

No public InfiniCore Python or C++ API changes are introduced. This consolidates the unique work from #1467, #1472, and #1476-#1479, and supersedes the earlier duplicate PRs #1468-#1471 and #1473.

The existing DeepSeek MLA paged-caching case with unequal key/value head sizes (576/512) remains unsupported by both the deprecated and canonical APIs. Extending that capability is outside this adapter-only migration.

Screenshots: N/A (backend adapter migration only).

Validation

Run on ssh nvidia in accelerator-dev/nvidia:latest on NVIDIA A100 GPUs:

  • clang-format 16.0.6 strict check passed for all 10 changed C++ files.
  • git diff --check origin/main..HEAD passed.
  • Unified InfiniOps build and xmake install _infinicore passed with legacy and canonical wrappers enabled together.
  • Paged caching standard equal-head-size suite: 24/24 passed across FP16/BF16/FP32 and both cache layouts.
  • Top-k softmax suite: 24/24 passed.
  • Sigmoid suite: 45/45 passed.
  • SiLU-and-mul suite: 36/36 passed.
  • Conv2d suite: 12/12 passed.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@voltjia