Skip to content

bpo-44032: Move data stack to thread from FrameObject. - #26076

Merged
markshannon merged 21 commits into
python:mainfrom
faster-cpython:datastack-on-thread
May 21, 2021
Merged

bpo-44032: Move data stack to thread from FrameObject.#26076
markshannon merged 21 commits into
python:mainfrom
faster-cpython:datastack-on-thread

Conversation

@markshannon

@markshannonmarkshannon commented May 12, 2021

Copy link
Copy Markdown
Member

Move local, cell and free variables, plus the evaluation stack to the thread.
Cuts down the size of frames to under 100 bytes, and enables further optimizations of Python-to-Python calls.

https://bugs.python.org/issue44032

@ericsnowcurrently
ericsnowcurrently self-requested a review May 17, 2021 16:31

@ericsnowcurrentlyericsnowcurrently left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks good and the value of this change seems pretty clear. 🙂

I'll probably take a second pass before approving. I mostly follow the change, but mapping the old code to the new conceptually isn't easy. The old code was fairly straight-forward to follow. The new code isn't quite so easy. It may help to have a bit more explanation somewhere of how the threadstate-based stack operates, including relative to frames.

PyObject *co_name; /* unicode (name, for reference) */
PyObject *co_linetable; /* string (encoding addr<->lineno mapping) See
Objects/lnotab_notes.txt for details. */
int co_nlocalsplus; /* Number of locals + free + cell variables */

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I've been calling this co_nfastlocals in my branches.

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it isn't all "fast" locals, as there are cells as well. Admittedly, co_nlocalsplus isn't a great name either.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"fastlocals" is what we already call them in ceval.c.

Comment threadInclude/cpython/frameobject.h
Comment threadInclude/cpython/frameobject.h Outdated
Comment threadPython/pystate.c Outdated
Comment threadInclude/internal/pycore_pymem.h
@bedevere-bot

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@ericsnowcurrentlyericsnowcurrently left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

After some offline discussion it makes sense now. We would still benefit from some explanation in the code though.

@markshannonmarkshannon added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 20, 2021
@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @markshannon for commit c5ccb7d 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 20, 2021
@markshannon

markshannon commented May 21, 2021

Copy link
Copy Markdown
MemberAuthor

The failure was a timeout on test_multiprocessing_fork one of the refleak buildbots.
The other refleak buildbots passed though, so there are no leaks (at least not that are caught by the buildbots).

@markshannon
markshannon merged commit b11a951 into python:mainMay 21, 2021
Comment threadObjects/frameobject.c
{
assert(f->f_own_locals_memory == 0);
assert(f->f_stackdepth == 0);
int size = frame_nslots(f);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markshannon, this gives a compiler error because frame_nslots() returns a Py_ssize_t.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@markshannon@bedevere-bot@ericsnowcurrently@gvanrossum@the-knights-who-say-ni