Wrong result 8bit blockwise quantization over float16

[ZiyueXu77]

System Info

Ubuntu 24.04

Reproduction

The following simple script, will yield all 0 dequantized results for all 1 inputs, and Process finished with exit code 139 (interrupted by signal 11:SIGSEGV)
`
import torch
from bitsandbytes.functional import quantize_blockwise, dequantize_blockwise

if name == "main":

test_torch = torch.ones([5, 10], dtype=torch.float16)

quantized, quantized_state = quantize_blockwise(test_torch)
absmax = quantized_state.absmax
code = quantized_state.code

dequantized = dequantize_blockwise(quantized, absmax=absmax, code=code)
print(dequantized)

dequantized = dequantize_blockwise(quantized, quant_state=quantized_state)
print(dequantized)

`

Expected behavior

torch.float32 will give correct results of all 1s with Process finished with exit code 0

[matthewdouglas]

Hi,
On cpu only torch.float32 is currently supported. It seems like we do not guard or warn against this.

[ZiyueXu77]

I see, thanks for the prompt response! will add the constraints in my pipeline

[ved1beta]

is this issue about adding warnings or adding float32 support ?

[matthewdouglas]

@ved1beta It's about adding support which will come with #1628 (targeting v0.47.0 release).

In #1544 we start to raise a RuntimeError and that will be in the v0.46.0 release:

RuntimeError: A must be float32 on cpu, got torch.float16

[matthewdouglas]

Closing as #1628 has been merged. We'll include it in a near-future release.