Skip to content

bpo-40077: Convert _csv module to use PyType_FromSpec - #20974

Closed
corona10 wants to merge 7 commits into
python:masterfrom
corona10:bpo-40077-csv
Closed

bpo-40077: Convert _csv module to use PyType_FromSpec #20974
corona10 wants to merge 7 commits into
python:masterfrom
corona10:bpo-40077-csv

Conversation

@corona10

@corona10corona10 commented Jun 19, 2020•
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

@corona10
corona10 requested a review from vstinnerJune 19, 2020 06:14
corona10
corona10 commented Jun 19, 2020
View reviewed changes

@corona10corona10 left a comment

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner
Please take a look

I locally tested with _csv module on subinterpreter and module test
And there was no leak.
(But import csv with subinterpreter still has leaks but not related to this PR)

@corona10

corona10 commented Jun 19, 2020

Copy link
Copy Markdown
MemberAuthor

FYI, macOS CI issue is not related to this PR

vstinner
vstinner reviewed Jun 19, 2020
View reviewed changes
Comment threadModules/_csv.c Outdated
Comment threadModules/_csv.c Outdated
Comment threadModules/_csv.c
Comment threadModules/_csv.c Outdated
@corona10
corona10 requested a review from vstinnerJune 19, 2020 10:39
corona10
corona10 commented Jun 19, 2020
View reviewed changes

@corona10corona10 left a comment

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vstinner

Thanks all of your reviews are applied and no memory leak was found!

vstinner
vstinner requested changes Jun 19, 2020
View reviewed changes

@vstinnervstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that it's possible to convert static types to heap types if they have Py_TPFLAGS_BASETYPE, since currently there is no way to retrieve the module from such type.

Comment threadModules/_csv.c
Comment threadModules/_csv.c
&strict))
return NULL;

_csvstate *state = PyType_GetModuleState(type);

@vstinnervstinnerJun 19, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyType_GetModuleState() is not safe if the type has Py_TPFLAGS_BASETYPE flag, which is the case here.

Is there a way to get the defining type in tp_new?

Comment threadModules/_csv.c
PyErr_Format(_csvstate_global->error_obj, "field larger than field limit (%ld)",
_csvstate_global->field_limit);
PyTypeObject *reader_type = Py_TYPE(self);
_csvstate *state = PyType_GetModuleState(reader_type);

@vstinnervstinnerJun 19, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Reader type has Py_TPFLAGS_BASETYPE: PyType_GetModuleState() is unsafe here.

Comment threadModules/_csv.c
return NULL;
}
self->dialect = (DialectObj *)_call_dialect(dialect, keyword_args);
_csvstate *state = get_csv_state(module);

@vstinnervstinnerJun 19, 2020

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

state can be moved at the beginning of the function, to avoid calling get_csv_state() twice.

@bedevere-bot

bedevere-bot commented Jun 19, 2020

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@vstinner

vstinner commented Jun 22, 2020

Copy link
Copy Markdown
Member

I understand that this PR (if merged) would fix https://bugs.python.org/issue14935

@corona10corona10 mentioned this pull request Sep 27, 2020
Merged
@encukouencukou mentioned this pull request Nov 10, 2020
Merged
@encukou

encukou commented Dec 15, 2020

Copy link
Copy Markdown
Member

The issue is now closed via #23224.
Thank you for working on it, though!

@encukouencukou closed this Dec 15, 2020
@corona10corona10 mentioned this pull request Jun 27, 2022
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinnervstinnervstinner requested changes

Assignees

No one assigned

Labels

awaiting changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@corona10@bedevere-bot@vstinner@encukou@the-knights-who-say-ni