Skip to content

gh-108494: Argument Clinic: fix support of Limited C API - #108536

Merged
serhiy-storchaka merged 7 commits into
python:mainfrom
serhiy-storchaka:clinic-limited-capi
Aug 28, 2023
Merged

gh-108494: Argument Clinic: fix support of Limited C API#108536
serhiy-storchaka merged 7 commits into
python:mainfrom
serhiy-storchaka:clinic-limited-capi

Conversation

@serhiy-storchaka

@serhiy-storchakaserhiy-storchaka commented Aug 27, 2023

Copy link
Copy Markdown
Member

I tried to enable the use of the limited C API in Argument Clinic for all code. With this PR it generates working code, and all tests are passed, except the low-level vectorcall test in test_call and few tests in test_clinic (as expected). _struct.c and winreg.c use converters incompatible with the limited C API. I temporary blacklisted these files. I only tested on Linux, so there may be errors in Windows or macOS specific modules.

@serhiy-storchaka

Copy link
Copy Markdown
MemberAuthor

Yet one issue: the TokenizerIter constructor is incompatible with PyArg_ParseTupleAndKeywords. I temporary worked it around by adding the default value for keyword-only extra_tokens parameter. Similar incompatibility was also a cause of failing two test_clinic tests. Resolving #74366 should fix this.

Comment threadLib/test/test_call.py

def test_varargs1min(self):
msg = r"get expected at least 1 argument, got 0"
msg = (r"get\(\) takes at least 1 argument \(0 given\)|"

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyArg_ParseTuple() produces slightly different error message.

Comment threadModules/_sqlite/module.c
Comment threadModules/_tracemalloc.c
@vstinner

Copy link
Copy Markdown
Member

I'm very excited by this change! I will try to review it soon.

Would it be possible to write unit tests in _testclinic_limited and test_clinic, for unusual cases like deprecated parameters and (if possible) defining class?

In my other PR, I wrote tests for basic positional or keywords parameters.

@serhiy-storchaka

Copy link
Copy Markdown
MemberAuthor

As for tests, I think that we should run all (or almost all) clinic tests in two modes. Instead of adding new tests for the limited API or duplicating test code we should generate and build different modules from the same source with different options. But this is a different PR.


arg = PyLong_AsInt(arg_);
if (arg == -1 && PyErr_Occurred()) {
if (!PyArg_Parse(arg_, "i:my_int_func", &arg)) {

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like regression, but actually not all inlined parsing code is compatible with the limited C API. In following PR I will implement inlining correctly, only for converters compatible with the limited C API. It will be large but mostly boring change (adding new parameter and simple checks in tens of functions), so I did not include it here.

Comment threadTools/clinic/clinic.py Outdated
@serhiy-storchaka
serhiy-storchaka requested a review from a team as a code ownerAugust 28, 2023 11:11
Comment threadTools/clinic/clinic.py Outdated
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Comment threadModules/_struct.c

@erlend-aaslanderlend-aasland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have missed something, but this looks good to me. Thanks, Serhiy!

@AlexWaygoodAlexWaygood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No complaints from me (though I'm not an expert on the limited C API)

@erlend-aaslanderlend-aasland changed the title gh-108494: Argument Clinic: fix support of limited C APIgh-108494: Argument Clinic: fix support of Limited C APIAug 28, 2023
@serhiy-storchaka
serhiy-storchaka merged commit bc5356b into python:mainAug 28, 2023
@serhiy-storchaka
serhiy-storchaka deleted the clinic-limited-capi branch August 28, 2023 13:04
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@serhiy-storchaka@vstinner@erlend-aasland@AlexWaygood@bedevere-bot