Skip to content

bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv - #21439

Closed
tontinton wants to merge 2 commits into
python:masterfrom
tontinton:fix-issue-41273
Closed

bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv#21439
tontinton wants to merge 2 commits into
python:masterfrom
tontinton:fix-issue-41273

Conversation

@tontinton

@tontintontontinton commented Jul 10, 2020

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue41270

I got about 120% better performance on await reader.read() using this branch.

The way I tested was writing a server / client:
server.py:

importasyncioimportcontextlibimportdatetimeasyncdefclient_connected(reader, writer):
start=datetime.datetime.now()
withcontextlib.closing(writer):
foriinrange(100000):
awaitreader.read(1024*64) # tweak this parameter as much as you likediff=datetime.datetime.now() -startprint(f'{diff.seconds}.{diff.microseconds}')
asyncdefmain():
server=awaitasyncio.start_server(client_connected, '127.0.0.1', 8888)
addr=server.sockets[0].getsockname()
print(f'Serving on {addr}')
asyncwithserver:
awaitserver.serve_forever()
if__name__=="__main__":
asyncio.run(main())

client.py:

importasyncioimportcontextlibasyncdefflood(ip, port):
message=b'A'*1024*128# tweak this parameter as much as you likereader, writer=awaitasyncio.open_connection(ip, port)
withcontextlib.closing(writer):
whileTrue:
writer.write(message)
awaitwriter.drain()
if__name__=="__main__":
asyncio.run(flood('127.0.0.1', 8888))

@tontintontontinton changed the title Fix issue 41273bpo-41270 asyncio: proactor read transport: better performance by using recv_into instead of recvJul 10, 2020
@tontintontontinton changed the title bpo-41270 asyncio: proactor read transport: better performance by using recv_into instead of recvbpo-41270: asyncio's proactor read transport's better performance by using recv_into instead of recvJul 10, 2020
@tontinton
tontintonforce-pushed the fix-issue-41273 branch 2 times, most recently from 96f0f5a to 9553dc1CompareJuly 10, 2020 22:37
By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.
This betters performance for any stream using proactor (basically any
asyncio stream on windows).
By doubling the read buffer size we get better performance.
@methanemethane changed the title bpo-41270: asyncio's proactor read transport's better performance by using recv_into instead of recvbpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recvJul 11, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

@tontinton@the-knights-who-say-ni@bedevere-bot