Skip to content

gh-121459: Deferred LOAD_ATTR (methods) - #124101

Closed
Fidget-Spinner wants to merge 8 commits into
python:mainfrom
Fidget-Spinner:deferred_method_final
Closed

gh-121459: Deferred LOAD_ATTR (methods)#124101
Fidget-Spinner wants to merge 8 commits into
python:mainfrom
Fidget-Spinner:deferred_method_final

Conversation

@Fidget-Spinner

@Fidget-SpinnerFidget-Spinner commented Sep 15, 2024•
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

@bedevere-appbedevere-appBot mentioned this pull request Sep 15, 2024
Closed
Fidget-Spinner
Fidget-Spinner commented Sep 15, 2024
View reviewed changes
Comment threadObjects/object.c
colesbury
colesbury reviewed Sep 16, 2024
View reviewed changes
Comment threadObjects/dictobject.c Outdated
Comment threadInclude/internal/pycore_stackref.h Outdated
Comment threadObjects/typeobject.c Outdated
Comment threadPython/bytecodes.c Outdated
Comment threadObjects/typeobject.c
Comment threadInclude/internal/pycore_dict.h Outdated
Comment threadObjects/dictobject.c Outdated
Comment threadObjects/dictobject.c Outdated
Comment threadObjects/object.c
Comment on lines +1625 to +1627
_PyObject_TryGetInstanceAttributeStackRef(obj, name, method)) {
if (!PyStackRef_IsNull(*method)) {
PyStackRef_XCLOSE(descr_st);

@colesburycolesburySep 16, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overwrites *method, but not descr_st, so descr_st may store a stack reference that's not visible to the GC. That seems fragile and potentially a problem.

@Fidget-SpinnerFidget-SpinnerSep 19, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to restore *method to descr_st after this in some cases (see below). I guess the safe thing would be to convert it to a strong ref?

@colesburycolesburyOct 29, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need both *method and descr_st if you refactor these function.

Alternatively take in a second _PyStackRef * to store the temporary values in a place that is visible to the GC.

@Fidget-SpinnerFidget-SpinnerNov 14, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the second approach needs to wait for @mpage 's PR on adding an extra stack slot?

@Fidget-SpinnerFidget-SpinnerNov 14, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh we don't, we can use self_or_null.

@Fidget-SpinnerFidget-SpinnerNov 14, 2024

Copy link
Copy Markdown
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok nevermind, this does indeed need Matt's fix. Because we unconditionally write to self_or_null now.

Comment threadObjects/object.c Outdated
@Fidget-Spinner

Fidget-Spinner commented Oct 29, 2024

Copy link
Copy Markdown
MemberAuthor

@colesbury gentle ping to respond to the review comments please

markshannon
markshannon reviewed Nov 14, 2024
View reviewed changes

@markshannonmarkshannon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to add a lot of complexity.
LOAD_ATTR is already the slow path, do you have performance numbers?
Without a significant speedup, this doesn't seem worth it.

For future reference, for large PRs like this, could you make sure there is specific issue where the change can be discussed and justified, rather adding them to a more general (and vague) issue like #121459

Comment threadInclude/internal/pycore_object.h
// This belongs here rather than pycore_stackref.h because including pycore_object.h
// there causes a circular include.
static inline _PyStackRef
_Py_TryXGetStackRef(PyObject **ptr)

@markshannonmarkshannonNov 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add clear comments as to what this does.
What is it "trying" to do? What happens if it fails? etc.

Comment threadInclude/internal/pycore_object.h
extern bool _PyObject_TryGetInstanceAttribute(PyObject *obj, PyObject *name,
PyObject **attr);

extern bool _PyObject_TryGetInstanceAttributeStackRef(PyObject *obj, PyObject *name,

@markshannonmarkshannonNov 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Likewise. Anything with "try" in the name needs to be carefully documented.

Comment threadInclude/internal/pycore_object.h
extern int _PyObject_IsAbstract(PyObject *);

PyAPI_FUNC(int) _PyObject_GetMethod(PyObject *obj, PyObject *name, PyObject **method);
PyAPI_FUNC(int) _PyObject_GetMethodStackRef(PyObject *obj, PyObject *name, _PyStackRef *method, _PyStackRef *spare);

@markshannonmarkshannonNov 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this have a "spare" parameter?

Comment threadInclude/internal/pycore_object.h

extern void _Py_GetConstant_Init(void);

extern void _PyType_LookupStackRef(PyTypeObject *type, PyObject *name, _PyStackRef *result);

@markshannonmarkshannonNov 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this fail? It should probably return an int indicating success/failure/error.

Comment threadPython/bytecodes.c
}

op(_LOAD_ATTR, (owner -- attr, self_or_null if (oparg & 1))) {
op(_LOAD_ATTR, (owner -- attr[1], self_or_null[1] if (oparg & 1))) {

@markshannonmarkshannonNov 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
op(_LOAD_ATTR, (owner--attr[1], self_or_null[1]if (oparg&1))) {
op(_LOAD_ATTR, (owner--attr, self_or_nullif (oparg&1))) {

You don't need the arrays. The code generator should spill the values to the in-memory stack if necessary.

Comment threadPython/compile.c
goto error;
}

/* Reserve an extra word on the stack to ensure there is space for uops to

@markshannonmarkshannonNov 14, 2024

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this necessary?

@Fidget-Spinner

Fidget-Spinner commented Nov 14, 2024

Copy link
Copy Markdown
MemberAuthor

@markshannon I'm tempted to wait for Matt to land his LOAD_ATTR specialization PR then take a look again. It may be possible that specialization makes this PR less useful.

@Fidget-Spinner

Fidget-Spinner commented Dec 23, 2024

Copy link
Copy Markdown
MemberAuthor

I'm closing this because #128164 addresses practically everything.

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@colesburycolesburycolesbury left review comments
@markshannonmarkshannonmarkshannon left review comments
@methanemethaneAwaiting requested review from methanemethane is a code owner
@iritkatrieliritkatrielAwaiting requested review from iritkatrieliritkatriel is a code owner

Assignees

No one assigned

Labels

awaiting core reviewskip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Fidget-Spinner@colesbury@markshannon