Skip to content

bpo-46315: Use fopencookie() to avoid dup() in _PyTokenizer_FindEncodingFilename - #32033

Merged
tiran merged 4 commits into
python:mainfrom
tiran:bpo-46315-fdopencookie
Mar 22, 2022
Merged

bpo-46315: Use fopencookie() to avoid dup() in _PyTokenizer_FindEncodingFilename#32033
tiran merged 4 commits into
python:mainfrom
tiran:bpo-46315-fdopencookie

Conversation

@tiran

@tirantiran commented Mar 21, 2022•
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

@pablogsalpablogsal added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 21, 2022
@bedevere-bot

bedevere-bot commented Mar 21, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 6be12fc 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 21, 2022
pablogsal
pablogsal reviewed Mar 21, 2022
View reviewed changes
Comment threadParser/tokenizer.c Outdated
// cookie is just the fd
borrowed b;
b.fd = fd;
return fopencookie(b.cookie, "r", cookie_io);

@pablogsalpablogsalMar 21, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fopencookie is a non-standard GNU extension. Is this available in all platforms targetted by web assembly? Does this need a configure check?

@tirantiranMar 21, 2022

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a configure check in my first version of the patch. I got rid of it because it makes no sense. Both WASI and Emscripten use musl libc as upper half of libc and it always provides fdopencookie.

I wish their was a more elegant way to solve the problem. _PyTokenizer_FindEncodingFilename uses a FILE pointer instead of a file descriptor.

@pablogsalpablogsalMar 22, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, I am honestly not very happy with the amount of extra code, but I understand why is needed.

@tiran

tiran commented Mar 22, 2022

Copy link
Copy Markdown
MemberAuthor

The failing tests on macOS ARM64 and FreeBSD are unrelated to my changes.

pablogsal
pablogsal reviewed Mar 22, 2022
View reviewed changes

@pablogsalpablogsal left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good to me, but I had no way to test it myself, so I am not formally approving. Feel free to land if you want though

pablogsal
pablogsal reviewed Mar 22, 2022
View reviewed changes
Comment threadMisc/NEWS.d/next/Core and Builtins/2022-03-21-20-05-10.bpo-46315.2QqFIC.rst Outdated
@tirantiran changed the title bpo-46315: Use fdopencookie() to avoid dup() in _PyTokenizer_FindEncodingFilenamebpo-46315: Use fopencookie() to avoid dup() in _PyTokenizer_FindEncodingFilenameMar 22, 2022
@tiran

tiran commented Mar 22, 2022

Copy link
Copy Markdown
MemberAuthor

I can shorten the code. fopencookie handles NULL function pointers correctly. Our code only needs a read callback.

@tiran
tiran merged commit 9b889b5 into python:mainMar 22, 2022
@tirantiran mentioned this pull request Oct 4, 2022
Closed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pablogsalpablogsalpablogsal left review comments
@lysnikolaoulysnikolaouAwaiting requested review from lysnikolaoulysnikolaou is a code owner

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tiran@bedevere-bot@pablogsal@the-knights-who-say-ni