Calling SelectorSocketTransport.writelines with a very large payload doesn't get completely written to the socket

[alisaifee]

Bug report

As a consequence of the optimization introduced in #91166, when SelectorSocketTransport.writelines is used with a very large buffer which can't be written in one shot either by using socket.send or socket.sendmsg the remaining data in the buffer is never written to the socket.

The following example can be used to reproduce the error:

import asyncio


async def send_message(chunk_size: int, chunks: int):
 await asyncio.sleep(1)
 reader, writer = await asyncio.open_connection("127.0.0.1", 9898)
 data = [bytes(chunk_size)] * chunks + [b"end"]
 writer.writelines(data)
 await writer.drain()
 writer.close()
 await writer.wait_closed()


async def handle_message(reader, writer):
 data = await reader.read()
 print(f"End marker: {data[-3:]!r}")
 writer.close()
 await writer.wait_closed()


async def run_server():
 server = await asyncio.start_server(handle_message, "127.0.0.1", 9898)
 async with server:
 await server.serve_forever()


async def main():
 await asyncio.gather(run_server(), send_message(6500, 100))


asyncio.run(main())

The example above prints End marker: b'end' with python 3.11 but not with python 3.12.0a6+

Your environment

• CPython versions tested on: 3.12.0a7
• Operating system and architecture: macOS 13.2

Linked PRs

• gh-103462: Ensure SelectorSocketTransport.writelines registers a writer when data is still pending #103463

[kumaraditya303]

Good catch! I have been running this code for a while but the buffers aren't large enough to trigger this.