Skip to content

GH-47625: [Python] Free-threaded musllinux and manylinux wheels started failing with cffi 2.0.0 - #47626

Merged
raulcd merged 9 commits into
apache:mainfrom
AlenkaF:fix-cffi-issues
Sep 29, 2025
Merged

GH-47625: [Python] Free-threaded musllinux and manylinux wheels started failing with cffi 2.0.0#47626
raulcd merged 9 commits into
apache:mainfrom
AlenkaF:fix-cffi-issues

Conversation

@AlenkaF

@AlenkaFAlenkaF commented Sep 23, 2025

Copy link
Copy Markdown
Member

Rationale for this change

Musllinux and manylinux jobs started failing on free-threaded build of CPython 3.13 due to cffi not supporting free-threaded CPython 3.13.

What changes are included in this PR?

Free-threaded 3.13 builds should use separate requirements with no cffi.

Are these changes tested?

Yes, with the extended builds.

Are there any user-facing changes?

No.

@github-actionsgithub-actionsBot added the awaiting review Awaiting review label Sep 23, 2025
@AlenkaFAlenkaF changed the title GH-47625: [Python] Free-threaded musllinux and manylinux jobs started failing with cffi 2.0.0GH-47625: [Python] Free-threaded musllinux and manylinux wheels started failing with cffi 2.0.0Sep 23, 2025
@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-manylinux-2-28-* wheel-musllinux-1-2-*

@github-actions

Copy link
Copy Markdown

Revision: 4f4e6e2

Submitted crossbow builds: ursacomputing/crossbow @ actions-23f202b4a9

TaskStatus
wheel-manylinux-2-28-cp310-cp310-amd64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-arm64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-amd64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-arm64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-amd64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-arm64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-amd64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-arm64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-amd64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-arm64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-amd64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-arm64GitHub Actions

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-manylinux-2-28-cp313-cp313t-amd64 wheel-musllinux-1-2-cp313-cp313t-amd64

@github-actions

Copy link
Copy Markdown

Revision: d617915

Submitted crossbow builds: ursacomputing/crossbow @ actions-c4d719cd10

TaskStatus
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-manylinux-2-28-cp313-cp313t-amd64 wheel-musllinux-1-2-cp313-cp313t-amd64

@github-actions

Copy link
Copy Markdown

Revision: 584c72c

Submitted crossbow builds: ursacomputing/crossbow @ actions-3e207c876e

TaskStatus
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-*

@github-actions

Copy link
Copy Markdown

Revision: 584c72c

Submitted crossbow builds: ursacomputing/crossbow @ actions-b957c597f8

TaskStatus
wheel-macos-monterey-cp310-cp310-amd64GitHub Actions
wheel-macos-monterey-cp310-cp310-arm64GitHub Actions
wheel-macos-monterey-cp311-cp311-amd64GitHub Actions
wheel-macos-monterey-cp311-cp311-arm64GitHub Actions
wheel-macos-monterey-cp312-cp312-amd64GitHub Actions
wheel-macos-monterey-cp312-cp312-arm64GitHub Actions
wheel-macos-monterey-cp313-cp313-amd64GitHub Actions
wheel-macos-monterey-cp313-cp313-arm64GitHub Actions
wheel-macos-monterey-cp313-cp313t-amd64GitHub Actions
wheel-macos-monterey-cp313-cp313t-arm64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-amd64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-arm64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-amd64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-arm64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-amd64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-arm64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-amd64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-arm64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-amd64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-arm64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-amd64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-arm64GitHub Actions
wheel-windows-cp310-cp310-amd64GitHub Actions
wheel-windows-cp311-cp311-amd64GitHub Actions
wheel-windows-cp312-cp312-amd64GitHub Actions
wheel-windows-cp313-cp313-amd64GitHub Actions
wheel-windows-cp313-cp313t-amd64GitHub Actions

@AlenkaF
AlenkaF marked this pull request as ready for review September 23, 2025 17:10
@AlenkaF

Copy link
Copy Markdown
MemberAuthor

Musllinux and manylinux free-threaded builds seem to be fixed by these changes.
cc @raulcd@pitrou

Comment threadci/scripts/python_wheel_unix_test.sh Outdated
Comment on lines +105 to +107
if [ "${FREE_THREADED}" == "OFF" ]; then
# Install testing dependencies
python -m pip install -U -r "${source_dir}/python/requirements-wheel-test.txt"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AlenkaF this is great. Should we remove the requirements installation on the docker file and install based on the new FREE_THREADED flag here?
I mean, remove the:
RUN python -m pip install --no-build-isolation -r /arrow/python/requirements-wheel-test-3.13t.txt
and install it on the python_wheel_unix_test.sh when "${FREE_THREADED}" == "ON" so we install requirements for both free-threaded and non free-threaded on the same place

@github-actionsgithub-actionsBot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Sep 23, 2025
Comment threadci/scripts/python_wheel_unix_test.sh Outdated
: "${CHECK_WHEEL_CONTENT:=ON}"
: "${CHECK_UNITTESTS:=ON}"
: "${INSTALL_PYARROW:=ON}"
: "${FREE_THREADED:=OFF}"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could perhaps use sysconfig instead of configuring this manually, as described here:
https://docs.python.org/3/howto/free-threading-python.html#identifying-free-threaded-python

@github-actionsgithub-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 24, 2025
@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-manylinux-2-28-cp313-cp313t-amd64 wheel-musllinux-1-2-cp313-cp313t-amd64

@github-actions

Copy link
Copy Markdown

Revision: 11490a4

Submitted crossbow builds: ursacomputing/crossbow @ actions-73b2c15461

TaskStatus
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions

# pandas doesn't provide wheels for aarch64 yet, so we have to install nightly Cython
# along with the rest of pandas' build dependencies and disable build isolation
COPY python/requirements-wheel-test.txt /arrow/python/
COPY python/requirements-wheel-test-3.13t.txt /arrow/python/

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if these will be available in the shell script so keeping the COPY in here for now.

@pitroupitrouSep 25, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they are, since the rest of python is.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, will remove this line as well.

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-manylinux-2-28-cp313-cp313t-amd64 wheel-musllinux-1-2-cp313-cp313t-amd64

@github-actions

Copy link
Copy Markdown

Revision: 792c5b9

Submitted crossbow builds: ursacomputing/crossbow @ actions-61eac3165e

TaskStatus
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-*

@github-actions

Copy link
Copy Markdown

Revision: 792c5b9

Submitted crossbow builds: ursacomputing/crossbow @ actions-bf9ca4eb45

TaskStatus
wheel-macos-monterey-cp310-cp310-amd64GitHub Actions
wheel-macos-monterey-cp310-cp310-arm64GitHub Actions
wheel-macos-monterey-cp311-cp311-amd64GitHub Actions
wheel-macos-monterey-cp311-cp311-arm64GitHub Actions
wheel-macos-monterey-cp312-cp312-amd64GitHub Actions
wheel-macos-monterey-cp312-cp312-arm64GitHub Actions
wheel-macos-monterey-cp313-cp313-amd64GitHub Actions
wheel-macos-monterey-cp313-cp313-arm64GitHub Actions
wheel-macos-monterey-cp313-cp313t-amd64GitHub Actions
wheel-macos-monterey-cp313-cp313t-arm64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-amd64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-arm64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-amd64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-arm64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-amd64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-arm64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-amd64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-arm64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-amd64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-arm64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-amd64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-arm64GitHub Actions
wheel-windows-cp310-cp310-amd64GitHub Actions
wheel-windows-cp311-cp311-amd64GitHub Actions
wheel-windows-cp312-cp312-amd64GitHub Actions
wheel-windows-cp313-cp313-amd64GitHub Actions
wheel-windows-cp313-cp313t-amd64GitHub Actions

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@pitrou@raulcd ready for another look!

@raulcdraulcd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @AlenkaF

@github-actionsgithub-actionsBot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Sep 25, 2025
Comment threadci/scripts/python_wheel_unix_test.sh Outdated
@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-manylinux-2-28-cp313-cp313t-amd64 wheel-musllinux-1-2-cp313-cp313t-amd64

@github-actions

Copy link
Copy Markdown

Revision: 67465c9

Submitted crossbow builds: ursacomputing/crossbow @ actions-7579e4a77b

TaskStatus
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions

@AlenkaF

Copy link
Copy Markdown
MemberAuthor

@github-actions crossbow submit wheel-*

@github-actions

Copy link
Copy Markdown

Revision: 67465c9

Submitted crossbow builds: ursacomputing/crossbow @ actions-2051105c13

TaskStatus
wheel-macos-monterey-cp310-cp310-amd64GitHub Actions
wheel-macos-monterey-cp310-cp310-arm64GitHub Actions
wheel-macos-monterey-cp311-cp311-amd64GitHub Actions
wheel-macos-monterey-cp311-cp311-arm64GitHub Actions
wheel-macos-monterey-cp312-cp312-amd64GitHub Actions
wheel-macos-monterey-cp312-cp312-arm64GitHub Actions
wheel-macos-monterey-cp313-cp313-amd64GitHub Actions
wheel-macos-monterey-cp313-cp313-arm64GitHub Actions
wheel-macos-monterey-cp313-cp313t-amd64GitHub Actions
wheel-macos-monterey-cp313-cp313t-arm64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-amd64GitHub Actions
wheel-manylinux-2-28-cp310-cp310-arm64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-amd64GitHub Actions
wheel-manylinux-2-28-cp311-cp311-arm64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-amd64GitHub Actions
wheel-manylinux-2-28-cp312-cp312-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313-arm64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-amd64GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-arm64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-amd64GitHub Actions
wheel-musllinux-1-2-cp310-cp310-arm64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-amd64GitHub Actions
wheel-musllinux-1-2-cp311-cp311-arm64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-amd64GitHub Actions
wheel-musllinux-1-2-cp312-cp312-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313-arm64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-amd64GitHub Actions
wheel-musllinux-1-2-cp313-cp313t-arm64GitHub Actions
wheel-windows-cp310-cp310-amd64GitHub Actions
wheel-windows-cp311-cp311-amd64GitHub Actions
wheel-windows-cp312-cp312-amd64GitHub Actions
wheel-windows-cp313-cp313-amd64GitHub Actions
wheel-windows-cp313-cp313t-amd64GitHub Actions

@AlenkaF
AlenkaF requested a review from pitrouSeptember 26, 2025 05:49
@raulcd
raulcd merged commit dc9753c into apache:mainSep 29, 2025
37 of 38 checks passed
@raulcdraulcd removed the awaiting merge Awaiting merge label Sep 29, 2025
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit dc9753c.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.

zanmato1984 pushed a commit to zanmato1984/arrow that referenced this pull request Oct 15, 2025
… started failing with cffi 2.0.0 (apache#47626)
### Rationale for this change
Musllinux and manylinux jobs started failing on free-threaded build of CPython 3.13 due to cffi not supporting free-threaded CPython 3.13.
### What changes are included in this PR?
Free-threaded 3.13 builds should use separate requirements with no cffi.
### Are these changes tested?
Yes, with the extended builds.
### Are there any user-facing changes?
No.
* GitHub Issue: apache#47625
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
Mottl pushed a commit to Mottl/arrow that referenced this pull request May 26, 2026
… started failing with cffi 2.0.0 (apache#47626)
### Rationale for this change
Musllinux and manylinux jobs started failing on free-threaded build of CPython 3.13 due to cffi not supporting free-threaded CPython 3.13.
### What changes are included in this PR?
Free-threaded 3.13 builds should use separate requirements with no cffi.
### Are these changes tested?
Yes, with the extended builds.
### Are there any user-facing changes?
No.
* GitHub Issue: apache#47625
Authored-by: AlenkaF <frim.alenka@gmail.com>
Signed-off-by: Raúl Cumplido <raulcumplido@gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@AlenkaF@raulcd@pitrou