Remove pre-Python 3.8 C code by rtobar · Pull Request #124 · python-cffi/cffi

rtobar · 2024-09-05T15:37:47Z

This is a bit of a big one, but it's mostly removal of dead code anyway, so hopefully it's not too controversial, and well received.

While doing some further code inspection today to re-tackle #86 I realised that there's still a fair bit of code in the C extension, and in the C extension generator, for Python < 3.8. Since 3.8 is the minimum supported version, it makes sense to remove that now-dead code in favour of a less complex codebase.

In order to make this a bit more digestible I split this work in two commits: one focuses on removing support for Python <3, and another for <3.8. Many of these changes are big #if blocks that could easily be removed. In a few places there were macros that were defined differently depending on the Python version, that would have now been left with a single definition. In these cases I removed the macro altogether in favour of using the direct Python C API or idiom as required (e.g., all the PyText_* macros are now PyUnicode_* calls, etc).

Both commits are (I think) correctly self-contained, at least locally they individually both compile, install and test correctly. So in case something breaks it should be easy to bisect back.

arigo · 2024-09-05T15:43:59Z

See #113 for my personal stance on this issue. @nitzmahone If wants to merge this anyway, go ahead.

rtobar · 2024-09-05T15:58:12Z

@arigo thanks for that input. I also maintain two libraries for which I kept 2.7 support until not too long ago, but finally decided to drop it, with the rationale being that 2.7 users still can use the old versions without lacking much functionality anyway. Kind-of-supporting 2.7 (or anything other than what setup.py states) will add a maintenance cost that officially you shouldn't need to pay.

I'm not sure if you're aware of this, but have a look at https://pypistats.org/packages/cffi. Seeing those numbers was also something that prompted me to drop support for <1% of users, who again can still download earlier versions of my library. Removing hundreds of lines of C code and macros I think pays the price.

My opinion only of course, if you decide you want to keep the code that's fine, you know your audience and needs better, I won't be pushing back.

nitzmahone · 2024-09-06T06:34:09Z

Yeah, especially with the very deep inspection and review of all the code that's going to be necessary for proper support of free-threaded Python builds, I'm all for minimizing compatibility code in the active development branches for Python versions that aren't supported/tested.

My experience has repeatedly taught me: if it's not getting tested, it's probably broken...

The old branches of course aren't going anywhere, so if someone really needs to make something work on a long-dead Python version, they can always install an older release or backport fixes to a private fork. We've had several firedrills in recent memory caused by "dead but still called" code breaking suddenly, so I'm quite happy to be more aggressive about cleaning that stuff up for 1.18 and beyond. Thanks!

rtobar · 2024-09-06T07:16:05Z

Thanks @nitzmahone, that sounds pretty good to me :)

I have been building some further changes on top of these in the hopes that they'd be merged. Based on your message I'll continue under that assumption.

Would you be happy for a similar effort in the python codebase?

rtobar · 2024-10-18T13:28:45Z

Gentle ping here to see if the intention of doing this clean-up is still there, thanks! I've just also rebased on top of the latest main branch.

rtobar · 2024-12-29T11:50:58Z

Another gentle ping after a few months of inactivity

mattip · 2025-06-25T05:55:35Z

I think it would be fine to remove pre-python3 from the src/c code and the vengine_cpy.py code for one PR. There is still a need to somewhat support Python-2 for PyPy-2, which still needs CFFI, even though PyPy vendors CFFI into its codebase. Once we do a first cleanup, we can see what is left.

mattip · 2025-06-25T05:59:52Z

Rereading #113, I think we should release 1.18 as the last version to still support Python2, and merge something like this PR together with the free-threading code for a new version (1.19? 2.0?). Then people who still wish to use CFFI on Python2 can pin it to 1.18

nitzmahone · 2025-06-25T16:19:32Z

I think we should release 1.18 as the last version to still support Python2, and merge something like this PR together with the free-threading code for a new version (1.19? 2.0?).

Is there actually anything unreleased or pending merge to main that's apropos for the older pypy branches? If not, the 1.17 branch is still there (and worst case, we could always backport whatever for a 1.17.2 in the future). IIUC the desire for a published pre-release with free-threaded support is the only time-sensitive item right now, so it'd be nice to avoid the churn of doing a 1.18 with nearly no changes and 1.19 with FT support concurrently.

nitzmahone · 2025-06-25T16:20:57Z

Another thought: for the pypy branches that would be orphaned by removal of old code from main- could their "current CFFI" checks be retargeted to the release-1.17 branch (as the last branch that would contain updates they could sync)? Then main going forward (which gets branched off for 1.18, 2.0, or whatever, don't care what we call it) can have old code ripped out at our leisure, and anything critical in the future that might be needed by the older pypy branches can just be backported to the 1.17 branch (with or without tagging/release, TBD).

arigo · 2025-06-25T17:20:41Z

My 2 cents: it would be nice if removing-Python-2-support and free-threading-support would both be released into cffi 2.0. It makes it clearer IMHO that it's the first couple of major changes to cffi that occurred for a long time. PyPy2 and users of old CPythons could be sticking with the latest cffi 1.x version. Future updates to cffi 2.0 would only need to be ported to PyPy3, which might remove a few hurdles. The jump to version 2.0 makes it clear when that change occurred.

mattip · 2025-06-26T05:08:41Z

it'd be nice to avoid the churn of doing a 1.18

OK, this is reasonable.

could their "current CFFI" checks be retargeted to the release-1.17 branch

Good idea.

It would be nice if removing-Python-2-support and free-threading-support would both be released into cffi 2.0.

We could do 2.0 with free-threaded support and partial Python2 cleanup (i.e. what is needed to make the merge of #178 easier) while declaring it no longer supports Python2. Then we could continue the cleanup in a 2.1. This would allow a faster release of the 2.0 free-threading support.

rtobar · 2025-06-27T13:28:00Z

I've been reading this these last messages with interest. Please let me know if you want me to rebase these changes on top of the latest main.

For those who haven't seen the detail, this PR has two commits:

The first removes C code for python 2
The second removes C code for python < 3.8

If we go ahead, do we want to:

Keep both commits
Declare 3.8 and unsupported, since that python version is discontinued. Maybe it's been removed already from cffi, I haven't checked.
Remove C code for python 3.8

mattip · 2025-06-28T05:52:01Z

I would prefer to separate the two commits. If these statistics can be believed, 3.8 still has about a 10% share on urllib3 downloads on PyPI

mattip · 2025-06-28T05:59:00Z

… although in the pyproject.toml there is already this which came in PR #129:

cffi/pyproject.toml

Line 15 in 67a170d

requires-python = ">=3.9"

@nitzmahone was that intentional?

mattip · 2025-12-01T07:33:16Z

2.0 has been released. There are conflicts here, it would be nice to clean this up.

rtobar · 2025-12-01T08:06:39Z

@mattip happy to, as long as there is a good level of certainty that these changes will be merged. They've been floating for 1.5 years now

mattip · 2025-12-01T08:13:11Z

Sure, I have a little bandwidth for this in the next week or so. Also there is #214.

rtobar · 2025-12-03T08:20:32Z

OK, rebased on latest main and force-pushed, at least no tests broke!

mattip

Some questions not really connected to this PR, the PR itself looks good

mattip · 2025-12-03T15:22:18Z

 &mb_as_buffer, /* tp_as_buffer */
- (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC |
- MINIBUF_TPFLAGS), /* tp_flags */
+ (Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC), /* tp_flags */


Could maybe get rid of parentheses around the flags

Parentheses removed 👍

mattip · 2025-12-03T15:30:51Z

-#endif
 if (bi == NULL)
 goto error;
 PyDict_SetItemString(result, "__builtins__", bi);


Not really part of this PR, since was always being done for python 3.0+, but I wonder why this is needed. Is __builtins__ still used somewhere inside CPython?

The docs in CPython say:
As an implementation detail, most modules have the name builtins made available as part of their globals. The value of builtins is normally either this module or the value of this module’s dict attribute. Since this is an implementation detail, it may not be used by alternate implementations of Python.
So looks unnecessary to me.

Maybe someday they will deprecate that, until then I guess it should stay.

mattip · 2025-12-03T15:46:40Z

LGTM. I will merge it in a day or two, to give time for others to take a look.

Most of this is removing code for PY_VERSION_MINOR < 3, removing the guards for PY_VERSION_MAJOR >= 3, and removing all unnecessary macros that would now have a single definition (e.g., PyText_Check -> PyUnicode_Check) in favour of using the direct Python C API for clarity. Only in minor circumstances some small logic needed to be changed. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

Similarly to the previous commit, the bulk of the work is removing code for PY_VERSION_HEX < 0x03080000, removing #if guards around PY_VERSION_HEX >= 0x03080000, and removing unnecessary macro definitions. Signed-off-by: Rodrigo Tobar <rtobar@icrar.org>

rtobar · 2025-12-03T15:49:49Z

Thanks @mattip, that's much appreciated

mattip · 2025-12-03T15:51:57Z

Please don't force push unless there is a problem that requires it, now I need to look at it all again.

rtobar · 2025-12-03T15:54:54Z

Ah, sorry, I didn't know that was the preference. The only change I made was the removal of parenthesis, you can still compare with https://github.com/python-cffi/cffi/compare/627ff54bffa4d9f83b894f8b5bc9f5ec902c2975..73f1185aa8a178f7502fda781cffb840aaff8650

mattip · 2025-12-03T16:06:34Z

you can still compare with

Ahh, cool, thanks.

mattip · 2025-12-08T11:36:49Z

Thanks @rtobar

nitzmahone mentioned this pull request Sep 6, 2024
Proper free-threaded ABI support blockers #126
Closed
11 tasks

nitzmahone added this to the 1.18 milestone Sep 6, 2024

rtobar force-pushed the remove-pre38-c-code branch from 8aa897d to db6983cCompare December 29, 2024 11:54

DimitriPapadopoulos mentioned this pull request Nov 22, 2025
Start removing pre-Python 3.8 code #214
Merged

mattip mentioned this pull request Dec 1, 2025
Manually apply ruff/flake8-implicit-str-concat rule ISC002 #215
Merged

rtobar force-pushed the remove-pre38-c-code branch from db6983c to 627ff54Compare December 3, 2025 08:01

mattip approved these changes Dec 3, 2025
View reviewed changes

rtobar added 2 commits December 3, 2025 23:49

rtobar force-pushed the remove-pre38-c-code branch from 627ff54 to 73f1185Compare December 3, 2025 15:49

mattip merged commit 239913d into python-cffi:mainDec 8, 2025
37 checks passed

rtobar deleted the remove-pre38-c-code branch December 8, 2025 11:42

DimitriPapadopoulos mentioned this pull request Dec 10, 2025
Keep removing Python pre-3.9 code #218
Draft

Uh oh!

Conversation

rtobar commented Sep 5, 2024

Uh oh!

arigo commented Sep 5, 2024

Uh oh!

rtobar commented Sep 5, 2024

Uh oh!

nitzmahone commented Sep 6, 2024

Uh oh!

rtobar commented Sep 6, 2024

Uh oh!

rtobar commented Oct 18, 2024• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rtobar commented Dec 29, 2024

Uh oh!

mattip commented Jun 25, 2025

Uh oh!

mattip commented Jun 25, 2025

Uh oh!

nitzmahone commented Jun 25, 2025

Uh oh!

nitzmahone commented Jun 25, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arigo commented Jun 25, 2025

Uh oh!

mattip commented Jun 26, 2025

Uh oh!

rtobar commented Jun 27, 2025

Uh oh!

mattip commented Jun 28, 2025

Uh oh!

mattip commented Jun 28, 2025

Uh oh!

mattip commented Dec 1, 2025

Uh oh!

rtobar commented Dec 1, 2025

Uh oh!

mattip commented Dec 1, 2025

Uh oh!

rtobar commented Dec 3, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattip left a comment

Choose a reason for hiding this comment

Uh oh!

mattipDec 3, 2025

Choose a reason for hiding this comment

Uh oh!

rtobarDec 3, 2025

Choose a reason for hiding this comment

Uh oh!

mattipDec 3, 2025

Choose a reason for hiding this comment

Uh oh!

rtobarDec 3, 2025

Choose a reason for hiding this comment

Uh oh!

mattipDec 3, 2025

Choose a reason for hiding this comment

Uh oh!

mattip commented Dec 3, 2025

Uh oh!

rtobar commented Dec 3, 2025

Uh oh!

mattip commented Dec 3, 2025

Uh oh!

rtobar commented Dec 3, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattip commented Dec 3, 2025

Uh oh!

Uh oh!

mattip commented Dec 8, 2025

Uh oh!

Reviewers

Assignees

Labels

rtobar commented Oct 18, 2024•
edited
Loading

nitzmahone commented Jun 25, 2025•
edited
Loading

rtobar commented Dec 3, 2025•
edited
Loading

rtobar commented Dec 3, 2025•
edited
Loading