Skip to content

bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv - #21442

Merged
1st1 merged 2 commits into
python:masterfrom
tontinton:fix-issue-41273
Jul 14, 2020
Merged

bpo-41273: asyncio's proactor read transport's better performance by using recv_into instead of recv#21442
1st1 merged 2 commits into
python:masterfrom
tontinton:fix-issue-41273

Conversation

@tontinton

@tontintontontinton commented Jul 11, 2020

Copy link
Copy Markdown
Contributor

https://bugs.python.org/issue41273

I got about 150% better performance on await reader.read() using this branch (Windows only).

The way I tested was writing a server / client:
server.py:

importasyncioimportcontextlibimportdatetimeasyncdefclient_connected(reader, writer):
start=datetime.datetime.now()
withcontextlib.closing(writer):
foriinrange(1000):
awaitreader.read(1024*64) # tweak this parameter as much as you likediff=datetime.datetime.now() -startprint(f'{diff.seconds}.{diff.microseconds}')
asyncdefmain():
server=awaitasyncio.start_server(client_connected, '127.0.0.1', 8888)
addr=server.sockets[0].getsockname()
print(f'Serving on {addr}')
asyncwithserver:
awaitserver.serve_forever()
if__name__=="__main__":
asyncio.run(main())

client.py:

importasyncioimportcontextlibasyncdefflood(ip, port):
message=b'A'*1024*4# tweak this parameter as much as you likereader, writer=awaitasyncio.open_connection(ip, port)
withcontextlib.closing(writer):
whileTrue:
writer.write(message)
awaitwriter.drain()
if__name__=="__main__":
asyncio.run(flood('127.0.0.1', 8888))

By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.
This betters performance for any stream using proactor (basically any
asyncio stream on windows).
By doubling the read buffer size we get better performance.
@1st1
1st1 merged commit 568fb0f into python:masterJul 14, 2020
@bedevere-bot

Copy link
Copy Markdown

@1st1: Please replace # with GH- in the commit message next time. Thanks!

@1st1

1st1 commented Jul 14, 2020

Copy link
Copy Markdown
Member

Thank you!

shihai1991 pushed a commit to shihai1991/cpython that referenced this pull request Aug 4, 2020
…using recv_into instead of recv (python#21442)
* bpo-41273: Proactor transport read loop to use recv_into
By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.
This betters performance for any stream using proactor (basically any
asyncio stream on windows).
* bpo-41273: Double proactor read transport buffer size
By doubling the read buffer size we get better performance.
shihai1991 pushed a commit to shihai1991/cpython that referenced this pull request Aug 20, 2020
…using recv_into instead of recv (python#21442)
* bpo-41273: Proactor transport read loop to use recv_into
By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.
This betters performance for any stream using proactor (basically any
asyncio stream on windows).
* bpo-41273: Double proactor read transport buffer size
By doubling the read buffer size we get better performance.
xzy3 pushed a commit to xzy3/cpython that referenced this pull request Oct 18, 2020
…using recv_into instead of recv (python#21442)
* bpo-41273: Proactor transport read loop to use recv_into
By using recv_into instead of recv we do not allocate a new buffer each
time _loop_reading calls recv.
This betters performance for any stream using proactor (basically any
asyncio stream on windows).
* bpo-41273: Double proactor read transport buffer size
By doubling the read buffer size we get better performance.
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants

@tontinton@bedevere-bot@1st1@kk456852@the-knights-who-say-ni