Skip to content

itertools.zip_longest use-after-free via re-entrant iterator exhaust #154672

Description

@tonghuaroot

Bug description

itertools.zip_longest has a use-after-free when one iterator's callback re-enters the zip_longest and exhausts a sibling iterator.

zip_longest_next_lock_held() borrows an iterator from lz->ittuple with PyTuple_GET_ITEM (no Py_INCREF), then calls PyIter_Next(it). If that call re-enters and drains the same iterator, the inner call does PyTuple_SET_ITEM(ittuple, i, NULL) + Py_DECREF(it). When the tuple holds the only reference, it is freed while the outer call still uses it.

The reproducer uses filter(None, ...) so the predicate calls bool(item), and del leaves the tuple as the sole owner of the filter:

importitertools, gczl_ref= [None]
classCustomIter:
def__init__(self, data):
self.data=dataself.idx=0def__iter__(self):
returnselfdef__next__(self):
ifself.idx>=len(self.data):
raiseStopIterationval=self.data[self.idx]
self.idx+=1returnvalclassEvil:
armed=Truedef__bool__(self):
ifEvil.armedandzl_ref[0] isnotNone:
Evil.armed=Falsetry:
whileTrue: # re-enter and drain the filter iteratornext(zl_ref[0])
exceptStopIteration:
passreturnFalsef=filter(None, CustomIter([Evil() for_inrange(3)]))
zl=itertools.zip_longest(f, range(5), fillvalue=-1)
zl_ref[0] =zldelf# ittuple now owns the only ref to the filtergc.collect()
print(next(zl))

Run with PYTHONMALLOC=debug python repro.py -> SIGSEGV. With the fix it runs fine.

CPython versions tested on

main

Operating systems tested on

macOS

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions