[CUDA] Build compact fpA intB kernels by default

[Tianlei Wu (tianleiwu)]

Description

• Build a compact fpA_intB CUDA kernel set by default: FP16 activations, INT4/INT8 weights, scale-only quantization, and block_size=32.
• Add onnxruntime_USE_FPA_INTB_GEMM_FULL to retain the complete BF16, zero-point, bias, larger-block-size, and native Hopper kernel matrix when required.
• Identify the compact configuration with USE_COMPACT_FPA_INTB_GEMM and exclude native Hopper kernels from it.
• Support the compact non-Hopper kernels on SM75 and newer devices, using SM75 below Ampere, SM89 on Ada, and the SM80 compatibility path otherwise.
• Keep eligibility, workspace estimation, profiling, prepacked-weight validation, tests, diagnostics, and documentation aligned with the compact contract.

Motivation and Context

The default fpA_intB build compiled many kernel variants that are not required by the FP16 INT4/INT8 RC model contract. A compact default reduces the generated CUDA kernel set while preserving the full matrix behind an explicit build option.

The compact kernels are non-Hopper kernels rather than SM80-only kernels: they support compute capability 7.5 and newer. Native Hopper TMA/WGMMA kernels remain restricted to SM90 and the full build.

Testing

• Compact CUDA provider and onnxruntime_provider_test build.
• Full-mode onnxruntime_providers_cuda_llm build.
• MatMulNBitsWorkspace.*: 14 tests passed.
• Fp16Int8GroupwiseTest.* and Fp16Int4GroupwiseTest.*: 2 tests passed on SM90 through the non-Hopper compatibility path.
• Changed-file lintrunner, Python syntax validation, and git diff --check passed.

[Copilot]

Pull request overview

Makes compact FP16 INT4/INT8 CUDA fpA_intB kernels the default while retaining the full kernel matrix behind an option.

Changes:

• Adds compact/full build configuration and source filtering.
• Aligns runtime eligibility, dispatch, profiling, validation, and workspace estimation.
• Updates tests and documentation for the compact contract.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
cmake/CMakeLists.txt	Adds compact and full build modes.
cmake/onnxruntime_cuda_source_filters.cmake	Filters unused compact-mode kernels.
docs/contrib_ops/cuda/matmul_nbits.md	Documents both kernel modes.
onnxruntime/contrib_ops/cuda/llm/fpA_intB_gemm/fpA_intB_gemm_template.h	Adds compact architecture dispatch.
onnxruntime/contrib_ops/cuda/llm/fpA_intB_gemv/dispatcher.h	Restricts compact GEMV variants.
onnxruntime/contrib_ops/cuda/llm/fpA_intB_gemv/fpA_intB_gemv.cu	Adds compact launch and support logic.
onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.cc	Implements compact eligibility and profiling.
onnxruntime/contrib_ops/cuda/quantization/matmul_nbits.h	Aligns construction and validation.
onnxruntime/test/contrib_ops/cuda_kernels/fpA_intB_gemm_kernel_test.cc	Exercises compact kernel shapes.
onnxruntime/test/contrib_ops/matmul_4bits_test.cc	Updates build-specific diagnostics.
onnxruntime/test/providers/cuda/test_cases/matmul_nbits_workspace_test.cc	Tests compact eligibility and architecture selection.
onnxruntime/test/python/quantization/test_op_matmulnbits_prepacked_cuda.py	Expands prepacked-weight parity coverage.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.