Pool timeout doesn't work

[karpetrosyan]

Discussion: #814

The pool timeout exception is now only raised when we are waiting for a connection from a pool and the pool is always full.

Problem

When the response releases the connection, all the requests in the queue can take the same connection.
But in the AsyncHTTP11Connection class, when the handle_async_request is called, it checks if his state is correct. If the state is not NEW or IDLE, that means that the other request has stolen that connection, and it raises ConnectionNotAvailable.

Like so:

httpcore/httpcore/_async/http11.py

Lines 78 to 84 in 8780c9c

	async with self._state_lock:
	if self._state in (HTTPConnectionState.NEW, HTTPConnectionState.IDLE):
	self._request_count += 1
	self._state = HTTPConnectionState.ACTIVE
	self._expire_at = None
	else:
	raise ConnectionNotAvailable()

Because the queued request has this cycle, the program may hang indefinitely.

• waiting for a connection
• gets the connection that is already in use (but the connection state has not been changed yet, so it thinks he is the only one who uses that connection)
• catches the ConnectionNotAvailable exception and tries again with the same pool timeout.

There is a code that can reproduce this issue (credits to @valsteen).

Server

import asyncio
from fastapi import FastAPI

app = FastAPI()

@app.get("/")
async def root():
 await asyncio.sleep(2)

Client

import time
import trio
from httpcore import AsyncConnectionPool

TIMEOUTS = {"read": 5, "write": 5, "pool": 5, "connect": 5}


async def read():
 start = time.time()
 try:
 response = await http.request(
 "GET", "http://127.0.0.1:8000", extensions={"timeout": TIMEOUTS}
 )
 if response.status // 100 != 2:
 raise RuntimeError
 except Exception as ex:
 print(f"failed: {ex.__class__}")

 print(f"time: {time.time()-start}")


async def main():
 async with trio.open_nursery() as nursery:
 for _ in range(100):
 nursery.start_soon(read)


if __name__ == "__main__":
 http = AsyncConnectionPool(
 max_connections=3,
 max_keepalive_connections=3,
 )
 trio.run(main)

[karpetrosyan]

@valsteen Would you like to open the pull request with the solution that you have shown in #814?

[valsteen]

@karpetrosyan great ! yes but I'll see how to make it 'proper'