Skip to content

gh-155389: Return bytes from _pyio.BytesIO.peek() - #155390

Open
fedonman wants to merge 2 commits into
python:mainfrom
fedonman:fix-pyio-bytesio-peek-bytes
Open

gh-155389: Return bytes from _pyio.BytesIO.peek()#155390
fedonman wants to merge 2 commits into
python:mainfrom
fedonman:fix-pyio-bytesio-peek-bytes

Conversation

@fedonman

@fedonmanfedonman commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

_pyio.BytesIO.peek() returned a slice of the internal bytearray, so it gave a bytearray where the C version and the docs say bytes. It also did not accept an __index__-able size, and did not take the lock while slicing.

It now does what read() does: coerce the size, take the lock, convert with take_bytes().

test_peek only used assertEqual, and bytearray(b'1') == b'1', so it passed either way. It now checks the type and an __index__ size, and fails without the change.

BytesIO.peek() is new in 3.16, so there is no NEWS entry.

peek() returned a slice of the internal bytearray, where read() converts with
take_bytes(). It also did not coerce its size through __index__ and did not
hold the lock while slicing, both of which read() and the C implementation do.
@cmaloney

Copy link
Copy Markdown
Contributor

cc: @marcelm@vstinner

Comment threadLib/_pyio.py Outdated
size = size_index()

with self._lock:
if size < 1:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the size piece here needs to be in the lock

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks. Only the slicing is under the lock now.

Comment threadLib/_pyio.py
return self._buffer[self._pos:self._pos + io.DEFAULT_BUFFER_SIZE]
return self._buffer[self._pos:self._pos + size]
try:
size_index = size.__index__

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that the size is ignored previously is actually intentional, that's the behavior in the C as well:

/*[clinic input]
@critical_section
_io._Buffered.peek
size: Py_ssize_t = 0
/
[clinic start generated code]*/
staticPyObject*
_io__Buffered_peek_impl(buffered*self, Py_ssize_tsize)
/*[clinic end generated code: output=ba7a097ca230102b input=56733376f926d982]*/
{
PyObject*res=NULL;
CHECK_INITIALIZED(self)
CHECK_CLOSED(self, "peek of closed file")
if (!ENTER_BUFFERED(self))
returnNULL;
if (self->writable) {
res=buffered_flush_and_rewind_unlocked(self);
if (res==NULL)
goto end;
Py_CLEAR(res);
}
res=_bufferedreader_peek_unlocked(self);
end:
LEAVE_BUFFERED(self)
returnres;
}

Copy link
Copy Markdown
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that one is _io._Buffered.peek (BufferedReader), which does ignore size. BytesIO.peek is in bytesio.c and uses it: size < 1 becomes DEFAULT_BUFFER_SIZE, then it clamps to what is left in the buffer.

io.BytesIO.peek(2) -> b'ab'
BufferedReader.peek(2) -> b'bcdef'

So the Python fallback should honour it too, which is what the __index__ coercion is for.

@aisk

aisk commented Aug 8, 2026

Copy link
Copy Markdown
Member

It's better to add a test to check if TypeError is raised for peek(1.5).

The size normalisation touches no shared state, so it does not need the lock.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@fedonman@cmaloney@aisk