gh-116146: Add C-API to create module from spec and initfunc by itamaro · Pull Request #139196 · python/cpython

itamaro · 2025-09-21T07:18:54Z

C-API WG decision: capi-workgroup/decisions#77

Issue: Expose a C-API function to allow custom importers to create a module using an init function #116146

encukou

This looks reasonable.

I'm proposing a replacement for Init functions in PEP-793, but that PEP adds PyModule_FromSlotsAndSpec and PyModule_Exec which should work for similar use cases.
Would it make sense to switch the spec & initfunc arguments to match that proposal?

I'm not sure if the word Builtin in the name is necessary -- is that to match the internal create_builtin? Would something like PyImport_LoadModuleFromInitfuncAndSpec work better?

Do you want to submit this to the C API WG?

itamaro · 2025-09-23T15:38:11Z

Would it make sense to switch the spec & initfunc arguments to match that proposal?

I think it should be fine! Is the PEP already implemented on main?

I'm not sure if the word Builtin in the name is necessary -- is that to match the internal create_builtin? Would something like PyImport_LoadModuleFromInitfuncAndSpec work better?

Yes, I used it to reflect what it does internally. PyImport_LoadModuleFromInitfuncAndSpec sounds great to me!

Do you want to submit this to the C API WG?

Will do. Should we first decide whether to change this to match PEP 793, or go to the WP with options and get their input?

encukou · 2025-09-24T13:55:55Z

Is the PEP already implemented on main?

No, it's not even accepted :)
I have a slight preference to align the two proposals, in case both make it. I'm checking if you have strong reasons for the order here. No need to update the PR now.

Will do. Should we first decide whether to change this to match PEP 793, or go to the WP with options and get their input?

Just go to the WG :)

itamaro · 2025-09-24T14:51:34Z

Just go to the WG :)

done :) capi-workgroup/decisions#77

- use `PyImport_CreateModuleFromInitfunc` for the new API - no need to introduce `create_builtin_ex` - update `create_builtin` and all callers - update exception message

- remove unused `found` variable - use `my_test_extension` instead of `embedded_ext` (the former is free-threading-ready, the latter prints a warning)

Co-authored-by: Kumar Aditya <kumaraditya@python.org>

encukou · 2025-11-11T15:03:49Z

I left my suggestions as a PR: itamaro#27

* Test single-phase init as well; don't use private APIs in test * Doc update --------- Co-authored-by: Itamar Oren <itamarost@gmail.com>

Co-authored-by: Victor Stinner <vstinner@python.org>

python-cla-bot · 2025-11-12T16:15:22Z

All commit authors signed the Contributor License Agreement.

…on under the free-threaded build

itamaro · 2025-11-12T16:20:26Z

thank you @encukou @vstinner @kumaraditya303 for the review and suggestions!

@encukou are you planning to merge gh-141197 soon? until that is merged, this PR has a docs failure due to referring to the function that you document in gh-141197.

vstinner

The new function PyImport_CreateModuleFromInitfunc() should be documented in Doc/whatsnew/3.15.rst.

Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Victor Stinner <vstinner@python.org>

vstinner

LGTM. Thanks for the updates!

vstinner · 2025-11-14T11:07:33Z

Congrats @itamaro!

…ythonGH-139196) Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>

bedevere-appBot mentioned this pull request Sep 21, 2025
Expose a C-API function to allow custom importers to create a module using an init function #116146
Closed

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from f9cd705 to 4363e5aCompare September 21, 2025 19:25

encukou mentioned this pull request Sep 23, 2025
_PyImport_SwapPackageContext would be helpful if exposed #139227
Closed

encukou reviewed Sep 23, 2025
View reviewed changes

Comment threadPython/import.c Outdated

itamaro mentioned this pull request Sep 24, 2025
Expose a C-API to allow custom importers to create a module directly from spec & initfunc capi-workgroup/decisions#77
Closed
6 tasks

vstinner reviewed Sep 26, 2025
View reviewed changes

Comment threadPython/import.c Outdated

itamaro added 5 commits November 8, 2025 08:09

pythongh-116146: Add new C-API to create builtin from spec and initfunc
403c7e2

Apply review feedback and CAPI WG decision
074a510
- use `PyImport_CreateModuleFromInitfunc` for the new API - no need to introduce `create_builtin_ex` - update `create_builtin` and all callers - update exception message

Add tests for the PyImport_CreateModuleFromInitfunc API
d38c0e4

Document the new PyImport_CreateModuleFromInitfunc C-API
d5be6e5

Add news entry
781a730

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from 4363e5a to 781a730Compare November 8, 2025 18:52

itamaro added 2 commits November 8, 2025 10:54

Fix lint
7af34eb

fix CI
35e9e78
- remove unused `found` variable - use `my_test_extension` instead of `embedded_ext` (the former is free-threading-ready, the latter prints a warning)

itamaro changed the title ~~gh-116146: Add new C-API to create builtin from spec and initfunc~~gh-116146: Add C-API to create module from spec and initfuncNov 9, 2025

itamaro marked this pull request as ready for review November 9, 2025 01:01

itamaro requested review from brettcannon, ericsnowcurrently, kumaraditya303, ncoghlan and warsaw as code owners November 9, 2025 01:01

bedevere-appBot added the awaiting review label Nov 9, 2025

kumaraditya303 reviewed Nov 9, 2025
View reviewed changes

Comment threadPython/import.c Outdated

Apply Kumar's review suggestion
6aa2fb2
Co-authored-by: Kumar Aditya <kumaraditya@python.org>

PyImport_CreateModuleFromInitfunc: Doc & test update (#27)
f652399
* Test single-phase init as well; don't use private APIs in test * Doc update --------- Co-authored-by: Itamar Oren <itamarost@gmail.com>

vstinner reviewed Nov 12, 2025
View reviewed changes

Comment threadDoc/c-api/import.rst Outdated
Comment threadInclude/cpython/import.h Outdated
Comment threadPrograms/_testembed.c Outdated
Comment threadPrograms/_testembed.c Outdated
Comment threadPrograms/_testembed.c Outdated
Comment threadPython/import.c

Apply Victor's review suggestions
c4f1cc8
Co-authored-by: Victor Stinner <vstinner@python.org>

itamaro added 2 commits November 12, 2025 08:17

Set error if initfunc is NULL
29b1b68

Account for the RuntimeWarning when import a singlephase init extensi…
1165950
…on under the free-threaded build

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from 8f16533 to 1165950Compare November 12, 2025 16:18

vstinner reviewed Nov 12, 2025
View reviewed changes

Comment threadMisc/NEWS.d/next/C_API/2025-11-08-10-51-50.gh-issue-116146.pCmx6L.rst Outdated

encukou reviewed Nov 13, 2025
View reviewed changes

Comment threadDoc/c-api/import.rst Outdated

kumaraditya303 reviewed Nov 13, 2025
View reviewed changes

Comment threadInclude/cpython/import.h Outdated

kumaraditya303 reviewed Nov 13, 2025
View reviewed changes

Comment threadPrograms/_testembed.c Outdated

kumaraditya303 reviewed Nov 13, 2025
View reviewed changes

Comment threadPython/import.c Outdated

Apply suggestions from code review
79e5d0e
Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Victor Stinner <vstinner@python.org>

itamaro requested a review from AA-Turner as a code owner November 13, 2025 15:31

Mention new C-API in whatsnew
fbfde0a

itamaro force-pushed the gh-116146-create-builtin-initfunc branch from fabafb5 to fbfde0aCompare November 13, 2025 15:32

vstinner approved these changes Nov 13, 2025
View reviewed changes

bedevere-appBot added awaiting merge and removed awaiting review labels Nov 13, 2025

kumaraditya303 approved these changes Nov 13, 2025
View reviewed changes

Merge branch 'main' intopythongh-116146-create-builtin-initfunc
f485fcb

encukou merged commit 1e4e59b into python:mainNov 14, 2025
46 checks passed

bedevere-appBot removed the awaiting merge label Nov 14, 2025

encukou added a commit to encukou/cpython that referenced this pull request Nov 14, 2025
Link reference to PyModule_Exec added inpython#139196
816023d

itamaro deleted the gh-116146-create-builtin-initfunc branch November 15, 2025 23:40

itamaro mentioned this pull request Dec 1, 2025
Assertion failures from calling create_builtin with invalid object #142029
Closed

itamaro mentioned this pull request Dec 11, 2025
gh-142029: Raise ModuleNotFoundError instead of crashing on nonexsistent module name given to create_builtin()#142054
Merged

Uh oh!

Uh oh!

Conversation

itamaro commented Sep 21, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

itamaro commented Sep 23, 2025

Uh oh!

encukou commented Sep 24, 2025

Uh oh!

itamaro commented Sep 24, 2025

Uh oh!

Uh oh!

Uh oh!

encukou commented Nov 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

python-cla-botBot commented Nov 12, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

itamaro commented Nov 12, 2025

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vstinner commented Nov 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

itamaro commented Sep 21, 2025•
edited
Loading

python-cla-botBot commented Nov 12, 2025•
edited
Loading