Integrate cffi-buildtool into CFFI itself by ngoldbaum · Pull Request #241 · python-cffi/cffi

ngoldbaum · 2026-04-20T19:42:36Z

Closes#47. Revives #76.

This adds a new submodule for cffi named cffi.buildtool and a new CLI script distributed by CFFI named gen-cffi-src. Also adds new tests and documentation. See the new documentation and tests for usage details.

I realize that this is a big change that makes a number of opinionated choices. I'm very happy to adjust things to suit the maintainers' taste.

My ultimate goal here is to make it possible for projects using CFFI to use arbitrary build tools and remove the perceived tight coupling between CFFI and setuptools/distutils. If there isn't appetite to add the code in cffi-buildtool to CFFI itself, I'd also happily document how to depend on and use cffi-buildtool instead.

See inklesspen/cffi-buildtool#2 where I got @inklesspen's blessing to do this.

The is based on code and documentation that were originally written by Rose Davidson (@inklesspen on GitHub) for the cffi-buildtool project: https://github.com/inklesspen/cffi-buildtool.

@inklesspen

This adds code and documentation that was originally written by Rose Davidson (@inklesspen on GitHub) for the cffi-buildtool project: https://github.com/inklesspen/cffi-buildtool

inklesspen · 2026-04-20T19:46:05Z

Just commenting here to confirm I give my full support for this upstreaming, and after a release is cut which includes these changes, I will mark my existing project as deprecated.

Also I'm available to answer any questions about the original code upon which this is based.

ngoldbaum

Some thoughts that occurred to me since yesterday and a typo fix

mattip · 2026-04-21T16:38:35Z

There is a clang-tsan failure

ngoldbaum · 2026-04-21T16:41:21Z

@mattip see #242 and #243, the failure is unrelated.

ngoldbaum · 2026-05-19T18:30:53Z

Sorry for taking so long to come back to this. I think this is ready for another round of review now.

mattip

This seems very helpful, and will provide a path to move away from distutils/setuptools only cffi support.

Some of my comments might miss the mark. In general I think the concept of "build" is not clear enough. A cffi out-of-line module has three parts: generate C code, compile the C code into a c-extension, and import the module. Does "build" address all three?

An in-line mode module (no compiler used) is different, I assume this does not address the inline-mode usage.

rgommers · 2026-06-08T20:15:05Z

I had a read through out of interest; overall this LGTM. The CLI taking the two main build modes and source as input, generated C code as output is useful and matches how Cython, f2py and other such codegen/binding tools are integrated into meson-python and scikit-build-core; the pattern should indeed work for other build backends that are able to build extension modules from C code as well.

Thanks for the review comments! I'm going to apply the review suggestions that are pending now, pull, and address the remaining comments. Co-authored-by: Matti Picus <matti.picus@gmail.com>

…ol CLI

ngoldbaum · 2026-06-09T20:19:35Z

I think I've responded to all the review comments. I'd appreciate another look :)

mattip

Sorry to be nit-picking but cffi is confusing anyway, and I think it is made more so by the use of generic “build” when a more specific concept is intended. As I understand it the intention of the tool is to expose the c-code generation mechanism provided by cffi. Then the c code is handed over to a Python compilation framework like meson or setuptools, and the resulting c-extension module can be packaged into a wheel by a wheel producing framework. I would be happier if the tool was called gen-cffi-code or gen-c-code and not buildtool. Additionally, it would be nice to scan for “build” and use a better word instead where possible.

Does that make sense or is it too restrictive?

mattip · 2026-06-20T09:31:40Z

+<https://scikit-build-core.readthedocs.io/>`_, or similar.
+
+Installing CFFI installs the ``gen-cffi-src`` script; running ``python
+-m cffi.buildtool`` invokes the same command line and behaves


I still find the name buildtool confusing. What does the module actually do? It does not replace mason/setuptools, does it do more than generate c code?

mattip · 2026-06-20T09:33:08Z

+----------------------------
+
+This mode takes a Python script that dynamically defines an FFI interface and
+accompanying C extension source code. The FFI definition script is the same


And does what exactly? What is the output?

mattip · 2026-06-20T10:07:06Z

 using
 ref
 cdef
+ buildtool


Can you change the references to not use the word “build”?
Suggested change
buildtool
gen-cffi-src

mattip · 2026-06-20T10:08:21Z

+interface. Two subcommands:
+
+``exec-python``
+ Execute a Python build script that constructs a :class:`cffi.FFI`


Suggested change
ExecuteaPythonbuildscriptthatconstructsa :class:`cffi.FFI`
ExecuteaPythonscriptthatconstructsa :class:`cffi.FFI`

mattip · 2026-06-20T10:09:10Z

+ describes) and emit the generated C source.
+
+``read-sources``
+ Build the :class:`cffi.FFI` from a separate ``cdef`` file and C


Suggested change
Buildthe :class:`cffi.FFI`fromaseparate``cdef``fileandC
Createthe :class:`cffi.FFI`fromaseparate``cdef``fileandC

rgommers · 2026-06-20T10:23:17Z

I would be happier if the tool was called gen-cffi-code or gen-c-code and not buildtool.

"buildtool" is the filename, the executable is now called gen-cffi-src. While that's a reasonable name: if that's being optimized, then I'd suggest starting with cffi (easier to identify). So cffi-gen-src or cffi-gen-sources/code.

Co-authored-by: Matti Picus <matti.picus@gmail.com>

ngoldbaum · 2026-06-23T20:27:28Z

I just pushed a commit that globally renames cffi.buildtool to cffi.gen_src and the CLI script from gen-cffi-src to cffi-gen-src. I also tried to remove references to builds and build scripts, replacing them with references to compilation steps that happen after cffi-gen-src finish and FFI definition scripts. I also tried to explain better that the FFI definition script should define an FFI object named ffibuilder by default and how to override that default.

mattip · 2026-06-24T05:22:56Z

Perfect. Thanks!

inklesspen · 2026-07-07T17:15:16Z

As I promised, I've marked cffi-buildtool (pypi, github) as deprecated

Integrate cffi-buildtool into CFFI itself
92345d6
This adds code and documentation that was originally written by Rose Davidson (@inklesspen on GitHub) for the cffi-buildtool project: https://github.com/inklesspen/cffi-buildtool

ngoldbaum mentioned this pull request Apr 20, 2026
Document how to use meson to build a cffi cextension #47
Closed

ngoldbaum commented Apr 21, 2026
View reviewed changes

Comment threaddoc/source/buildtool.rst Outdated
Comment threaddoc/source/cffi-gen-src.rst
Comment threadtesting/cffi1/buildtool_example2/src/csrc/square.c Outdated

ngoldbaum added 3 commits May 19, 2026 11:45

Merge branch 'main' into integrate-buildtool
7f161ff

typo fix
9293037

Use symlinks and give test folders better names
5dad224

ngoldbaum mentioned this pull request Jun 8, 2026
Initial support for abi3t.abi3 wheels. #232
Merged

mattip reviewed Jun 8, 2026
View reviewed changes

rgommers reviewed Jun 8, 2026
View reviewed changes

Comment threaddoc/source/buildtool.rst Outdated

rgommers reviewed Jun 8, 2026
View reviewed changes

Comment threaddoc/source/buildtool.rst Outdated

rgommers reviewed Jun 8, 2026
View reviewed changes

Comment threaddoc/source/cdef.rst Outdated

rgommers reviewed Jun 8, 2026
View reviewed changes

Comment threadsrc/cffi/buildtool/_gen.py Outdated

ngoldbaumand others added 4 commits June 9, 2026 11:42

Apply suggestions from code review
e25ff27
Thanks for the review comments! I'm going to apply the review suggestions that are pending now, pull, and address the remaining comments. Co-authored-by: Matti Picus <matti.picus@gmail.com>

Remove Python interface and consolidate functionality in cffi.buildto…
dec64ee
…ol CLI

Respond to code review comments
6df3f77

stop referring to 'build scripts'
f8e2c2e

Restore the console script
50a29be

mattip reviewed Jun 20, 2026
View reviewed changes

ngoldbaumand others added 3 commits June 23, 2026 13:03

Apply suggestions from code review
54945bd
Co-authored-by: Matti Picus <matti.picus@gmail.com>

Respond to Matti and Ralf's renaming suggestions
dd1b7fb

Merge branch 'main' into integrate-buildtool
26b3d3a

mattip merged commit 26b3d3a into python-cffi:mainJun 24, 2026
38 checks passed

This was referenced Jun 24, 2026
Python 3.15 limited API extensions #226
Closed
Support for declarative project configuration (setup.cfg / pyproject.toml) #55
Closed

mgorny mentioned this pull request Jul 7, 2026
testing/cffi1/test_cffi_gen_src_meson.py::test_meson_python_build don't work offline #255
Closed

	ExecuteaPythonbuildscriptthatconstructsa :class:`cffi.FFI`
	ExecuteaPythonscriptthatconstructsa :class:`cffi.FFI`

	Buildthe :class:`cffi.FFI`fromaseparate``cdef``fileandC
	Createthe :class:`cffi.FFI`fromaseparate``cdef``fileandC

Uh oh!

Conversation

ngoldbaum commented Apr 20, 2026• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

inklesspen commented Apr 20, 2026

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mattip commented Apr 21, 2026

Uh oh!

ngoldbaum commented Apr 21, 2026

Uh oh!

ngoldbaum commented May 19, 2026

Uh oh!

mattip left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rgommers commented Jun 8, 2026

Uh oh!

ngoldbaum commented Jun 9, 2026

Uh oh!

mattip left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattipJun 20, 2026

Choose a reason for hiding this comment

Uh oh!

mattipJun 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mattipJun 20, 2026

Choose a reason for hiding this comment

Uh oh!

mattipJun 20, 2026

Choose a reason for hiding this comment

Uh oh!

mattipJun 20, 2026

Choose a reason for hiding this comment

Uh oh!

rgommers commented Jun 20, 2026

Uh oh!

ngoldbaum commented Jun 23, 2026

Uh oh!

mattip commented Jun 24, 2026

Uh oh!

Uh oh!

inklesspen commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ngoldbaum commented Apr 20, 2026•
edited
Loading