GH-47256: [Python] Do not use cffi in free-threaded 3.13 builds by AlenkaF · Pull Request #47313 · apache/arrow

AlenkaF · 2025-08-12T08:51:49Z

Rationale for this change

Ubuntu and Windows wheels started failing on free-threaded build of CPython 3.13 due to cffi not supporting free-threaded CPython 3.13.

What changes are included in this PR?

Free-threaded 3.13 builds use separate requirements with no cffi.

Are these changes tested?

Yes, with the extended builds.

Are there any user-facing changes?

No.

GitHub Issue: [Python] Free-threaded wheels for 3.13 have started failing with cffi 2.0.0b1 #47256

github-actions · 2025-08-12T08:52:16Z

⚠️ GitHub issue #47256has been automatically assigned in GitHub to PR creator.

AlenkaF · 2025-08-12T08:52:34Z

@github-actions crossbow submit wheel-windows-cp313-cp313t-amd64

github-actions · 2025-08-12T08:54:44Z

Revision: fcd62fe215030e748ae65def07e68ebea69b6035

Submitted crossbow builds: ursacomputing/crossbow @ actions-4c2b829fab

Task	Status
wheel-windows-cp313-cp313t-amd64

AlenkaF · 2025-08-12T08:55:13Z

@github-actions crossbow submit test-ubuntu-22.04-python-313-freethreading

github-actions · 2025-08-12T08:57:30Z

Revision: fcd62fe215030e748ae65def07e68ebea69b6035

Submitted crossbow builds: ursacomputing/crossbow @ actions-860e036ce5

Task	Status
test-ubuntu-22.04-python-313-freethreading

AlenkaF · 2025-08-12T09:23:12Z

Installing cffi again with a pinned version won't help, as the first install from requirements-*-test.txt already triggers the failure.

Pinning cffi globally isn’t ideal since this only affects the 3.13 free-threaded build.

Instead, we can patch the requirements file directly in the Dockerfile to exclude cffi and re-add it with a pinned version (<2.0.0). This avoids maintaining a separate override file and keeps the change localized to the affected build.

AlenkaF · 2025-08-12T09:35:24Z

@github-actions crossbow submit test-ubuntu-22.04-python-313-freethreading

github-actions · 2025-08-12T09:37:38Z

Revision: d853ccc9ba0371906bec98d56e2641a16f955199

Submitted crossbow builds: ursacomputing/crossbow @ actions-5fc39b3bfa

Task	Status
test-ubuntu-22.04-python-313-freethreading

AlenkaF · 2025-08-12T10:09:57Z

@github-actions crossbow submit wheel-windows-cp313-cp313t-amd64

github-actions · 2025-08-12T10:12:09Z

Revision: d853ccc9ba0371906bec98d56e2641a16f955199

Submitted crossbow builds: ursacomputing/crossbow @ actions-2685669349

Task	Status
wheel-windows-cp313-cp313t-amd64

AlenkaF · 2025-08-12T10:27:59Z

The last commit fixes the build failures related to cffi and the CPython 3.13 version discrepancy.

I’m not sure if patching in the Dockerfile is the best approach, so I’m happy to change it if needed—cc @kou.

Now that the cffi failure is resolved, I’m continuing to investigate the Windows Cython failures (see #47308). If I find a fix, I might add it here in the same PR.

AlenkaF · 2025-08-12T11:17:21Z

@github-actions crossbow submit wheel-windows-*

github-actions · 2025-08-12T11:19:34Z

Revision: cc758f4d36e3fdc48c96a85bc82a77b1bbf0d15e

Submitted crossbow builds: ursacomputing/crossbow @ actions-3c0c8c4c06

Task	Status
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp39-cp39-amd64

AlenkaF · 2025-08-12T11:39:14Z

Ah, I need to fix the Windows patch issue. Didn't saw that before: https://github.com/ursacomputing/crossbow/actions/runs/16905769938/job/47895087244#step:9:504

AlenkaF · 2025-08-12T12:01:39Z

@github-actions crossbow submit wheel-windows-*

github-actions · 2025-08-12T12:03:58Z

Revision: caa7067a6fa956364bc905b0c3d185c205c603bd

Submitted crossbow builds: ursacomputing/crossbow @ actions-02fe533f65

Task	Status
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp39-cp39-amd64

AlenkaF · 2025-08-12T13:01:44Z

@github-actions crossbow submit wheel-windows-cp310-cp310-amd64

github-actions · 2025-08-12T13:07:01Z

Revision: 55dc4acfcb1f9decc6921276295ce745a90f03cf

Submitted crossbow builds: ursacomputing/crossbow @ actions-b37f110292

Task	Status
wheel-windows-cp310-cp310-amd64

AlenkaF · 2025-08-12T15:28:23Z

@github-actions crossbow submit wheel-windows-cp310-cp310-amd64

github-actions · 2025-08-12T15:30:38Z

Revision: d9e53fd1786daf7a085c2054252c23b27c8b9d75

Submitted crossbow builds: ursacomputing/crossbow @ actions-1131ef4771

Task	Status
wheel-windows-cp310-cp310-amd64

AlenkaF · 2025-08-12T18:36:13Z

@github-actions crossbow submit wheel-windows-cp310-cp310-amd64

github-actions · 2025-08-12T18:38:28Z

Revision: 4228c421ce036d96ea2eb947ec2dff1c1ddefac3

Submitted crossbow builds: ursacomputing/crossbow @ actions-bee8b2d046

Task	Status
wheel-windows-cp310-cp310-amd64

kou · 2025-08-12T22:26:35Z

Can we use Environment Markers https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers instead of this approach? Something like:
--- a/python/requirements-test.txt+++ b/python/requirements-test.txt@@ -1,4 +1,5 @@-cffi+cffi; python_version != '3.13t'+cffi<2.0.0; python_version == '3.13t' hypothesis packaging pandas
(I'm not sure whether we have a variable to determine free-threaded CPython...)

Oh, that is much much nicer! Thank you @kou .
I noticed a use of PYTHON_GIL in the free-threaded builds. Looking at python-wheels/github.osx.yml it seems same as what you suggested in this diff.

Unfortunately, python_version doesn't support detailed versions like 3.13t — it only handles the major.minor format:
Ignoring cffi: markers 'python_version == "3.13t"' don't match your environment
See: https://github.com/ursacomputing/crossbow/actions/runs/16930331373/job/47974180274#step:6:722
Other markers listed in PEP 508 don't help either.
Using something like os.environ.get("PYTHON_GIL") in a requirements file is also ignored by pip, since arbitrary Python expressions aren’t supported in markers.
Not sure about other options — will keep investigating.

That's surprising that it only handles major and minor versions. I think the docs even show examples with a patch version?
https://packaging.python.org/en/latest/specifications/dependency-specifiers/#specification
I wonder if including a patch number here would be helpful? I'm not sure that Python's packaging story is complete around the t suffix and how version comparisons are handled (my knowledge could also be outdated!)

It is surprising, yeah. In the examples shared patch versions are not used in the python_version Environment Marker. For example, this issue led me to believe that python_version only considers the major and minor versions.
As for the t suffix, I’m also not entirely sure about the background there. I vaguely remember hearing at EuroPython that it's going to stay, but I could be misremembering 😬

pitrou · 2025-08-27T08:42:14Z

+# version. However, there is no need to make this strictly the oldest version,
+# so it can be broadened to have a single version specification across platforms.
+# (`~=x.y.z` specifies a compatible release as `>=x.y.z, == x.y.*`)
+numpy~=1.21.3; python_version < "3.11"


Since this file is for 3.13, these lines can be simplified :)

Oh yes! Will do.

Done: 430f26b

pitrou · 2025-08-27T08:42:39Z

@AlenkaF Can you update the PR title to reflect the PR contents?

AlenkaF · 2025-09-01T06:07:03Z

@github-actions crossbow submit test-ubuntu-22.04-python-313-freethreading

github-actions · 2025-09-01T06:09:20Z

Revision: 430f26b

Submitted crossbow builds: ursacomputing/crossbow @ actions-c0a94d79aa

Task	Status
test-ubuntu-22.04-python-313-freethreading

raulcd · 2025-09-01T13:43:18Z

@github-actions crossbow submit test-ubuntu-22.04-python-313-freethreading

github-actions · 2025-09-01T13:45:33Z

Revision: c5c8e4e

Submitted crossbow builds: ursacomputing/crossbow @ actions-ac6d583157

Task	Status
test-ubuntu-22.04-python-313-freethreading

AlenkaF · 2025-09-01T16:37:19Z

Should we wait for the Windows wheels issue to be fixed so we can test the changes on wheel-windows-cp313-cp313t-amd64 also?

raulcd · 2025-09-02T07:48:06Z

Should we wait for the Windows wheels issue to be fixed so we can test the changes on wheel-windows-cp313-cp313t-amd64 also?

I don't think we should, the Windows wheel issue is currently "blocked" by vcpkg issues and not being able to properly diagnose due to the lack of logs from vcpkg.

raulcd

LGTM

AlenkaF · 2025-09-02T07:51:31Z

@pitrou mind having one last look before we merge?

pitrou

This LGTM, thank you.

In the future, we might want to replace this with a requirements-test-free-threaded.txt...

conbench-apache-arrow · 2025-09-02T17:03:56Z

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 55be8c0.

There weren't enough matching historic benchmark results to make a call on whether there were regressions.

The full Conbench report has more details.

ngoldbaum · 2025-09-12T15:53:37Z

Sorry the explicit lack of support for 3.13t in CFFI 2.0.0 caused you all a little headache here. Glad it was all sorted out eventually.

AlenkaF · 2025-09-15T06:41:20Z

No worries at all, just a minor PR needed. Thanks for all the work on your side @ngoldbaum !

…apache#47313) ### Rationale for this change Ubuntu and Windows wheels started failing on free-threaded build of CPython 3.13 due to cffi not supporting free-threaded CPython 3.13. ### What changes are included in this PR? Free-threaded 3.13 builds use separate requirements with no cffi. ### Are these changes tested? Yes, with the extended builds. ### Are there any user-facing changes? No. * GitHub Issue: apache#47256 Authored-by: AlenkaF <frim.alenka@gmail.com> Signed-off-by: AlenkaF <frim.alenka@gmail.com>

github-actionsBot added the awaiting review Awaiting review label Aug 12, 2025

AlenkaF mentioned this pull request Aug 12, 2025
[Python] Nightly wheels missing for Windows #47308
Closed

AlenkaF changed the title ~~GH-47256: [Python] Pin cffi in free-threaded builds~~GH-47256: [Python] Pin cffi in free-threaded builds and use newer Win SDKAug 12, 2025

AlenkaF changed the title ~~GH-47256: [Python] Pin cffi in free-threaded builds and use newer Win SDK~~GH-47256: [Python] Pin cffi in free-threaded builds and use newer Win SDK in Windows wheelsAug 12, 2025

github-actionsBot added the Component: Python label Aug 12, 2025

kou reviewed Aug 12, 2025
View reviewed changes

github-actionsBot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Aug 12, 2025

github-actionsBot added the awaiting changes Awaiting changes label Aug 27, 2025

pitrou reviewed Aug 27, 2025
View reviewed changes

AlenkaF changed the title ~~GH-47256: [Python] Pin cffi in free-threaded builds~~GH-47256: [Python] Do not use cffi in free-threaded 3.13 buildsAug 27, 2025

AlenkaF mentioned this pull request Aug 27, 2025
[CI][Python] Update win wheels and free-threaded build for Python 3.14 #47437
Closed

Simplify requirements
430f26b

github-actionsBot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Sep 1, 2025

Reorganise requirements-wheel-test-3.13t
c5c8e4e

raulcd approved these changes Sep 2, 2025
View reviewed changes

github-actionsBot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Sep 2, 2025

pitrou approved these changes Sep 2, 2025
View reviewed changes

AlenkaF merged commit 55be8c0 into apache:mainSep 2, 2025
34 of 36 checks passed

AlenkaF removed the awaiting merge Awaiting merge label Sep 2, 2025

AlenkaF mentioned this pull request Sep 2, 2025
[Python] Free-threaded wheels for 3.13 have started failing with cffi 2.0.0b1 #47256
Closed

AlenkaF deleted the fix-windows-builds branch September 2, 2025 07:56

AlenkaF mentioned this pull request Sep 23, 2025
[Python] Free-threaded musllinux and manylinux jobs started failing with cffi 2.0.0 #47625
Closed

ryancheley mentioned this pull request Sep 26, 2025
Add Python 3.13 free-threaded and Python 3.14 support to CI ryancheley/yt-cli#616
Closed
4 tasks

Uh oh!

Conversation

AlenkaF commented Aug 12, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

AlenkaF commented Aug 12, 2025

Uh oh!

github-actionsBot commented Aug 12, 2025

Uh oh!

kouAug 12, 2025

Choose a reason for hiding this comment

Uh oh!

AlenkaFAug 13, 2025

Choose a reason for hiding this comment

Uh oh!

AlenkaFAug 13, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

WillAydAug 15, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AlenkaFAug 18, 2025

Choose a reason for hiding this comment

Uh oh!

pitrouAug 27, 2025

Choose a reason for hiding this comment

Uh oh!

AlenkaFAug 27, 2025

Choose a reason for hiding this comment

Uh oh!

AlenkaFSep 1, 2025

Choose a reason for hiding this comment

Uh oh!

pitrou commented Aug 27, 2025

AlenkaF commented Aug 12, 2025•
edited
Loading

AlenkaFAug 13, 2025•
edited
Loading

WillAydAug 15, 2025•
edited
Loading

AlenkaF commented Sep 15, 2025•
edited
Loading