GH-132554: "Virtual" iterators

[markshannon]

Just a draft PR until I have performance numbers.

• Issue: Use tagged ints for faster iteration #132554

[markshannon]

Performance is good. Nothing amazing, but a small speedup.

Stats show no significant changes

[Fidget-Spinner]

@markshannon I don't think it matters here, but you didn't update in https://github.com/python/cpython/pull/132545/filesPy_TAG_BITS to 3. It's still 1. I know it "technically" doesn't matter because we use a bitshift to zero it out, but we should still update it to be consistent.

[markshannon]

@markshannon I don't think it matters here, but you didn't update in https://github.com/python/cpython/pull/132545/filesPy_TAG_BITS to 3. It's still 1. I know it "technically" doesn't matter because we use a bitshift to zero it out, but we should still update it to be consistent.

Fixed in #134244

[iritkatriel]

I don't understand why you use this condition. Should it not be assert(ref.bits + 4 > ref.bits) or something like that?

[iritkatriel]

Would it help to define these via a macro? Something like

#define STACKREF_CHECK_FUNC(T) \
static inline bool \
PyStackRef_ ## T ## Check(_PyStackRef stackref) \
if (PyStackRef_IsTaggedInt(stackref)) { \
return false; \
} \
return Py ## T ## _Check(PyStackRef_AsPyObjectBorrow(stackref)); \
}
...
STACKREF_CHECK_FUNC(Exception);
STACKREF_CHECK_FUNC(Code);
STACKREF_CHECK_FUNC(Function);

[Fidget-Spinner]

I think it might not work when want to define variants of Check and CheckExact, though we can always define two macros for that if we go down this route.

[iritkatriel]

I'd add this to make sure the test is testing what the comment is saying.
assert(sys.getrefcount(x) == 1)

[iritkatriel]

Do here same as in d9dc597 ?