[cuda backend] int5 quantization support - #20757

Merged

Gasoonjia merged 20 commits into

mainfrom

Jul 9, 2026

Commits on Jun 28, 2026

[cuda] Compact int4/int6 weight quant metadata (bf16 -> uint8 + per-row super-scale)
Gasoonjia
committed
7c758dd

Commits on Jun 30, 2026

Merge branch 'main' into cuda-int4-int6-metadata-opt
Gasoonjia
authored
ad5ce4d

Commits on Jul 1, 2026

[cuda] Update int4/int6 plain_mm tests for compact quant metadata API
Gasoonjia
committed
6aa4c57
lint
Gasoonjia
committed
e50dfe5

Commits on Jul 2, 2026

Commits on Jul 6, 2026

Commits on Jul 7, 2026

Merge branch 'main' into cuda-int4-int6-metadata-opt
Gasoonjia
authored
7560591

Commits on Jul 9, 2026

solve comment
Gasoonjia
committed
c7b7f9a
Merge branch 'main' into cuda-int4-int6-metadata-opt
Gasoonjia
authored
210cb79
[cuda] Fix black f-string regex parse in gen_plain_mm_test_vectors (concat form for pre-PEP701 black)
Gasoonjia
committed
f39b17f
[cuda] Add genuine INT5 (GGUF Q5_K) weight pipeline
Gasoonjia
committed
c103618
[cuda][int5] per-256 fp16 scale+zero step packed with z_pack (mirror int4): +1.52dB SNR, op-level tests
Gasoonjia
committed
3bf3528
lint
Gasoonjia
committed
0f7ac75
remove note
Gasoonjia
committed
9051581
lint
Gasoonjia
committed
7282d6f
[cuda][int5] Apply int4/int6 review conventions to int5
Gasoonjia
committed
4bea7ac
lint
Gasoonjia
committed
81cf4f1
Merge branch 'main' into gguf-cuda-int5
Gasoonjia
authored
a232ee8