Connection reuse in HTTP/2 on concurrent requests

[yeraydiazdiaz]

While doing some tests with HTTP/2 and trace logging I noticed that when using concurrent requests connections are being created until hitting max_connections, even though (I believe) only one should be created and new streams should be used within the same connection.

Here's a quick example:

import asyncio

import httpx


async def main(url, n):
 async with httpx.AsyncClient(
 pool_limits=httpx.PoolLimits(soft_limit=2, hard_limit=5),
 timeout=httpx.Timeout(5.0),
 http2=True,
 ) as client:
 await asyncio.gather(*[request(client, url, i) for i in range(n)])


async def request(client, url, i):
 response = await client.get(url)
 assert response.status_code == 200
 assert response.http_version == "HTTP/2"


if __name__ == "__main__":
 asyncio.run(main("https://example.org", 10))

With some logging tweaks prints:

DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=2
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=3
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=4
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=4
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=4
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5
DEBUG [2020-05-10 10:36:32] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=5

Tracing the flow in the code I found that:

• The first coro will attempt to fetch a connection for an origin
• It will return None so one is created
• It will attempt to add to the pool
• But the asyncio.wait_for acquire will not return immediately and will trigger an event loop switch
• The next coroutine will come in and try to pick a connection from the pool
• Will not find one and will create it and so on

And sure enough, removing the wait_for the output is:

DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - created connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - get_connection_from_pool=(b'https', b'example.org', 443)
DEBUG [2020-05-10 10:43:35] httpcore._async.connection_pool - reuse connection=AsyncHTTPConnection origin=(b'https', b'example.org', 443) http2=True state=0 pool_size=1

Note a single connection is created and the following coroutines reuse it.

Obviously just removing the wait_for is not the right thing to do since we want to keep the pooling timeout, but we probably need to think of a different way to achieve it to maximize throughput in HTTP/2.