index arrays get converted to int32 from CVXPY's converters

[Transurgeon]

I am creating this issue because we noticed this while working on the kron atom converter in this PR: cvxpy/cvxpy#3447.

This is just an issue to start a discussion on if it's something to worry about/change in the future.
Otherwise we can probably close this.

[Transurgeon]

also for an index to be more than int32, this would mean there are over 2 billion entries in the matrix. I think at this point we probably have other things to worry about?

[PTNobel]

It's worth digging through the CVXPY history, but we have had issues before re 32 bit ints. I don't really remember the details

[PTNobel]

It might not be relevant for this usecase, but I think we had a canonicalized problem where there was overflow

[Transurgeon]

It might not be relevant for this usecase, but I think we had a canonicalized problem where there was overflow

I remember something similar!

I ended up digging it up here: cvxpy/cvxpy@9a2b6d0.
It was caused by divmod of the shape (i.e. size of m*n, recall that's what shape[0] refers to and shape[1] is the number of parameters) with n_cols (i.e. number of variables).

So I think we should be relatively safe for now, since the fix above would be needed whenever m*n > 2 billion, regardless of the number of nonzeros in the matrix. But good point to bring it up.
I think it could be something to be careful of in the future, certainly.

[Transurgeon]

It might not be relevant for this usecase, but I think we had a canonicalized problem where there was overflow

I remember something similar!

I ended up digging it up here: cvxpy/cvxpy@9a2b6d0. It was caused by divmod of the shape (i.e. size of m*n, recall that's what shape[0] refers to and shape[1] is the number of parameters) with n_cols (i.e. number of variables).

So I think we should be relatively safe for now, since the fix above would be needed whenever m*n > 2 billion, regardless of the number of nonzeros in the matrix. But good point to bring it up. I think it could be something to be careful of in the future, certainly.

I could be remembering a different case than the one you are suggesting though.
Let's definitely come back to this once I ask you to review the diffengine backend.