Add support for python 3.13 & remove various version caps

[dbaxa]

Add support for python 3.13 & remove various version caps

[dbaxa]

For some reason 01e998c#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711 reverted the change made in d690516 to remove the version caps.

[jprakash-db]

@dbaxa Python3.13 testing can be included. As far as relaxing the numpy version we reverted it as it had compatibility issues with pandas - https://pandas.pydata.org/docs/whatsnew/v2.2.2.html#pandas-2-2-2-is-now-compatible-with-numpy-2-0

[dbaxa]

@jprakash-db can you provide more details about the compatibility issues? In our use case we didn't see any problems with using python 3.13 pandas 2.2.3 & numpy 2.2.3 when using a modified version of databricks-sql-python locally.

As in

One major caveat is that arrays created with numpy 2.0's new StringDtype will convert to object dtyped arrays upon :class:Series/:class:DataFrame creation. Full support for numpy 2.0's StringDtype is expected to land in pandas 3.0.

doesn't seem like an issue to me because StringDtype was introduced in numpy 2.0.0 and people apparently would need to explicitly use the new type (pandas-dev/pandas#58104 (comment) & aws/sagemaker-python-sdk#4882 (comment)).

[dhirschfeld]

As databricks-sql-python doesn't directly import numpy we shouldn't be specifying it as a dependency here.

In this code base we only rely on pandas functionality, we don't care how pandas implements that functionality under the hood.

Our contract is with the public API of pandas and we trust the pandas devs to ensure their library works correctly. If pandas doesn't work with a modern numpy it is for them to pin numpy as it is their dependency.

pandas claims to support numpy>=1.26:
https://github.com/pandas-dev/pandas/blob/main/pyproject.toml#L30-L31

...so if that is not the case we should open an issue (or PR) in the pandas repo to apply the pin there.

Packaging bugs occur (all the time!) but if this pin here is to work around a mis-specified dependency that pandas once had at one time, the cure is worse that the cold. Pinning to an old numpy means that databricks-sql-connector can't be used in any modern analytics environment which isn't great for a package designed to support analytics.

[samikshya-db]

Thanks for raising this : dhirschfeld, this makes sense to me. I am closing this in favour of #510