Supported data types

[kgryte]

This issue seeks to come to a consensus on the minimum set of data types an array library must support in order to conform to the specification.

Prior Art

Supported data types across array libraries...

• NumPy

bool_
bool8
byte
short
intc
int_
longlong
intp
int8
int16
int32
int64
ubyte
ushort
uintc
uint
ulonglong
uintp
uint8
uint16
uint32
uint64
half
single
double
float_
longfloat
float16
float32
float64
float96
float128
csingle
complex_
clongfloat
complex64
complex128
complex192
complex256
object_
bytes_
unicode_
void

• PyTorch

bfloat16
bool
complex64
complex128
float16
float32
float64
int8
int16
int32
int64
uint8

• Tensorflow

bool
bfloat16
complex64
complex128
float16
float32
float64
int16
int32
int64
qint8
qint16
qint32
quint8
quint16
string
uint8
uint16
uint32
uint64

• JAX

bool
bfloat16
complex64
complex128
float16
float32
float64
int8
int16
int32
int64
uint8
uint16
uint32
uint64

• CuPy

bool_
complex64
complex128
float16
float32
float64
int8
int16
int32
int64
uint8
uint16
uint32
uint64

• Dask (see NumPy)

• MXNet (see NumPy)

• PyData/Sparse (see NumPy)

Proposal

This issue proposes to specify that all specification conforming array libraries must, at minimum, support the following data types:

bool

int8
int16
int32
int64

uint8
uint16
uint32
uint64

float32
float64

The above data types are common across all array libraries considered in prior art (with PyTorch being the exception).

Notes

• complex64 and complex128 are currently omitted from this proposal, as I'd like to defer consideration of some of the thornier aspects of how complex numbers are handled for future specification iterations. The proposed types have considerable prior art and are well-established, and, when questions arise regarding their behavior, normative references, such as IEEE 754 for floating-point arithmetic, are available.

[kgryte]

Based on feedback, I've removed float16 from the proposed list of data types.

bfloat16 is a potential future candidate, given its widespread usage among accelerator libraries; however, remains unclear whether array libraries such as NumPy would ever add built-in support. Accordingly, I have not included bfloat16 as part of the minimum subset which all specification-conforming libraries must adhere to.

Re: complex64 and complex128. Some suggestion that complex number data types could be included as part of specification "extensions" (i.e., for array libraries which do support complex number data types, here are required conventions, behavior, and associated interfaces). But still some debate whether complex number data types should be required for all specification-conforming array libraries.

[rgommers]

bfloat16 is a potential future candidate, given its widespread usage among accelerator libraries; however, remains unclear whether array libraries such as NumPy would ever add built-in support.

I've searched the NumPy issue tracker and mailing list for bfloat16, it hasn't been discussed before. Took a temperature reading, there's some reluctance to add it; at the very least it should start as an external package, which will then need some changes once the big dtype refactor currently ongoing is complete, and then if it's stable and used it needs a NEP (NumPy Enhancement Proposal) for adding to numpy itself.

[kgryte]

My sense is that we should postpone consideration of bfloat16 until after v1. It is currently supported by 3 out of 8 array libraries (see OP). While useful, at least for v1, would be good to coalesce around what is already widely supported to ensure maximum adoption. Once we've achieved that goal, we can begin nudging array libraries to standardizing and implementing, in this case, emerging data types.

[rgommers]

This was all agreed on, and that bfloat16, complex64 and complex128 are for a future version of the standard is noted in the Scope section. So closing this.