GH-43846: [Python][Packaging] Remove numpy dependency from pyarrow packaging by raulcd · Pull Request #44148 · apache/arrow

raulcd · 2024-09-17T11:11:30Z

WIP

Creating the PR to validate if / what are the CI failures

Rationale for this change

Once the issue has been merged:

[Python] Make NumPy an optional runtime dependency #25118

We do not require numpy as a pyarrow dependency.

What changes are included in this PR?

Remove numpy as a required dependency

Are these changes tested?

Via CI

Are there any user-facing changes?

Yes, pyarrow won't install numpy as a required dependency.

GitHub Issue: [Python][Packaging] Remove numpy required dependency from pyarrow packaging #43846

…row packaging

github-actions · 2024-09-17T11:11:55Z

⚠️ GitHub issue #43846has been automatically assigned in GitHub to PR creator.

raulcd · 2024-09-17T14:17:43Z

@github-actions crossbow submit -g python

github-actions · 2024-09-17T14:20:28Z

Revision: 2a52bce

Submitted crossbow builds: ursacomputing/crossbow @ actions-929cd4e6ac

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-cython2
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

raulcd · 2024-09-17T15:13:38Z

@github-actions crossbow submit -g wheel

raulcd · 2024-09-17T15:13:57Z

@github-actions crossbow submit verify-python

github-actions · 2024-09-17T15:16:22Z

Revision: 2a52bce

Submitted crossbow builds: ursacomputing/crossbow @ actions-6a782cdfa6

Task	Status
verify-rc-source-python-linux-almalinux-8-amd64
verify-rc-source-python-linux-conda-latest-amd64
verify-rc-source-python-linux-ubuntu-20.04-amd64
verify-rc-source-python-linux-ubuntu-22.04-amd64
verify-rc-source-python-macos-amd64
verify-rc-source-python-macos-arm64
verify-rc-source-python-macos-conda-amd64

github-actions · 2024-09-17T15:16:38Z

Revision: 2a52bce

Submitted crossbow builds: ursacomputing/crossbow @ actions-09c6d448ff

Task	Status
python-sdist
wheel-macos-monterey-cp310-amd64
wheel-macos-monterey-cp310-arm64
wheel-macos-monterey-cp311-amd64
wheel-macos-monterey-cp311-arm64
wheel-macos-monterey-cp312-amd64
wheel-macos-monterey-cp312-arm64
wheel-macos-monterey-cp313-amd64
wheel-macos-monterey-cp313-arm64
wheel-macos-monterey-cp39-amd64
wheel-macos-monterey-cp39-arm64
wheel-manylinux-2-28-cp310-amd64
wheel-manylinux-2-28-cp310-arm64
wheel-manylinux-2-28-cp311-amd64
wheel-manylinux-2-28-cp311-arm64
wheel-manylinux-2-28-cp312-amd64
wheel-manylinux-2-28-cp312-arm64
wheel-manylinux-2-28-cp313-amd64
wheel-manylinux-2-28-cp313-arm64
wheel-manylinux-2-28-cp39-amd64
wheel-manylinux-2-28-cp39-arm64
wheel-manylinux-2014-cp310-amd64
wheel-manylinux-2014-cp310-arm64
wheel-manylinux-2014-cp311-amd64
wheel-manylinux-2014-cp311-arm64
wheel-manylinux-2014-cp312-amd64
wheel-manylinux-2014-cp312-arm64
wheel-manylinux-2014-cp313-amd64
wheel-manylinux-2014-cp313-arm64
wheel-manylinux-2014-cp39-amd64
wheel-manylinux-2014-cp39-arm64
wheel-windows-cp310-amd64
wheel-windows-cp311-amd64
wheel-windows-cp312-amd64
wheel-windows-cp313-amd64
wheel-windows-cp39-amd64

raulcd · 2024-09-19T09:41:51Z

@jorisvandenbossche is this something we want to do before 18.0.0?

jorisvandenbossche · 2024-09-20T16:55:15Z

Should we bring that up on the mailing list?

My guess is that it won't be that disruptive because most people will have numpy installed via other dependencies anyway, but it might be a bit hard to predict.
If we want to do it at some point, I think we can as well do it for 18.0

jorisvandenbossche · 2024-10-02T15:20:22Z

Looks good!

Can you update the documentation? https://arrow.apache.org/docs/dev/python/install.html#dependencies

…d dependencies

raulcd · 2024-10-03T14:25:20Z

Can you update the documentation? https://arrow.apache.org/docs/dev/python/install.html#dependencies

Thanks @jorisvandenbossche I've just updated the Numpy dependency from required to optional on the docs

raulcd · 2024-10-09T13:25:10Z

@jorisvandenbossche if you are ok with this I'll merge for 18.0.0

jorisvandenbossche · 2024-10-09T16:20:56Z

I am fine with that

raulcd

👍

…ckaging (#44148) ## WIP Creating the PR to validate if / what are the CI failures ### Rationale for this change Once the issue has been merged: - #25118 We do not require numpy as a pyarrow dependency. ### What changes are included in this PR? Remove numpy as a required dependency ### Are these changes tested? Via CI ### Are there any user-facing changes? Yes, pyarrow won't install numpy as a required dependency. * GitHub Issue: #43846 Authored-by: Raúl Cumplido <raulcumplido@gmail.com> Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>

conbench-apache-arrow · 2024-10-10T17:37:44Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 8be5f9c.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 4 possible false positives for unstable benchmarks that are known to sometimes produce them.

With Arrow 18.0.0 numpy is not a dependency anymore: apache/arrow#44148 I think it would be good to also remove it from PyIceberg

* Remove numpy as a hard dependency With Arrow 18.0.0 numpy is not a dependency anymore: apache/arrow#44148 I think it would be good to also remove it from PyIceberg * Add link to issue

apacheGH-43846: [Python][Packaging] Remove numpy dependency from pyar…
2a52bce
…row packaging

github-actionsBot added Component: Python awaiting committer review Awaiting committer review labels Sep 17, 2024

raulcd marked this pull request as ready for review October 2, 2024 15:13

raulcd requested a review from jorisvandenbossche October 2, 2024 15:20

Update docs to move Numpy to optional dependencies instead of Require…
54b99e1
…d dependencies

github-actionsBot added the Component: Documentation label Oct 3, 2024

jorisvandenbossche approved these changes Oct 9, 2024
View reviewed changes

github-actionsBot added awaiting merge Awaiting merge and removed awaiting committer review Awaiting committer review labels Oct 9, 2024

raulcd commented Oct 10, 2024
View reviewed changes

raulcd merged commit 8be5f9c into apache:mainOct 10, 2024

raulcd removed the awaiting merge Awaiting merge label Oct 10, 2024

raulcd mentioned this pull request Oct 10, 2024
[Python][Packaging] Remove numpy required dependency from pyarrow packaging #43846
Closed

github-actionsBot added the awaiting changes Awaiting changes label Oct 10, 2024

raulcd deleted the GH-43846 branch October 10, 2024 10:02

raulcd removed the awaiting changes Awaiting changes label Oct 10, 2024

bigluck mentioned this pull request Oct 28, 2024
Bump PyArrow to 18.0.0 apache/iceberg-python#1256
Merged

Fokko mentioned this pull request Oct 30, 2024
[Python] Add np.arange equivalent #44583
Closed

Fokko added a commit to Fokko/iceberg-python that referenced this pull request Oct 30, 2024
Remove numpy as a hard dependency
db2be16
With Arrow 18.0.0 numpy is not a dependency anymore: apache/arrow#44148 I think it would be good to also remove it from PyIceberg

Fokko added a commit to Fokko/iceberg-python that referenced this pull request Oct 30, 2024
Remove numpy as a hard dependency
500da55
With Arrow 18.0.0 numpy is not a dependency anymore: apache/arrow#44148 I think it would be good to also remove it from PyIceberg

Fokko mentioned this pull request Oct 30, 2024
Remove numpy as a hard dependency apache/iceberg-python#1270
Merged

This was referenced Jan 8, 2025
Arrow~=18.1.0 G-Research/JollyJack#54
Merged
Arrow ~=18.1.0 G-Research/PalletJack#76
Merged

Uh oh!

Conversation

raulcd commented Sep 17, 2024• edited by github-actions BotLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WIP

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actionsBot commented Sep 17, 2024

Uh oh!

raulcd commented Sep 17, 2024

Uh oh!

github-actionsBot commented Sep 17, 2024

Uh oh!

raulcd commented Sep 17, 2024

Uh oh!

raulcd commented Sep 17, 2024

Uh oh!

github-actionsBot commented Sep 17, 2024

Uh oh!

github-actionsBot commented Sep 17, 2024

Uh oh!

raulcd commented Sep 19, 2024

Uh oh!

jorisvandenbossche commented Sep 20, 2024

Uh oh!

jorisvandenbossche commented Oct 2, 2024

Uh oh!

raulcd commented Oct 3, 2024

Uh oh!

raulcd commented Oct 9, 2024

Uh oh!

jorisvandenbossche commented Oct 9, 2024

Uh oh!

raulcd left a comment

Choose a reason for hiding this comment

Uh oh!

conbench-apache-arrowBot commented Oct 10, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

raulcd commented Sep 17, 2024•
edited by github-actions Bot
Loading