Skip to content

Drop RuntimeError on closing a connection pool with active connections. - #631

Merged
lovelydinosaur merged 6 commits into
masterfrom
drop-runtime-error-on-closing-connection-pool-with-open-connections
Dec 14, 2022
Merged

Drop RuntimeError on closing a connection pool with active connections.#631
lovelydinosaur merged 6 commits into
masterfrom
drop-runtime-error-on-closing-connection-pool-with-open-connections

Conversation

@lovelydinosaur

@lovelydinosaurlovelydinosaur commented Nov 30, 2022

Copy link
Copy Markdown
Contributor

Closes#564.

Closes#561 (The motivation given there becomes redundant. Errors will be raised against the connections themselves)

With this script...

example.py:

importanyioimporthttpcoreasyncdefworker(client, tg):
r=awaitclient.request('HEAD', 'https://google.com')
print(r)
tg.cancel_scope.cancel() # Cancel our other outstanding requestsasyncdefmain():
asyncwithhttpcore.AsyncConnectionPool() asclient, anyio.create_task_group() astg:
for_inrange(3):
tg.start_soon(worker, client, tg)
if__name__=="__main__":
anyio.run(main)

Before this change...

$ venv/bin/python ./example.py <Response [301]>
Traceback (most recent call last):
File "./example.py", line 18, in<module>
anyio.run(main)
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 292, in run
returnnative_run(wrapper(), debug=debug)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
returnfuture.result()
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "./example.py", line 14, in main
tg.start_soon(worker, client, tg)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/connection_pool.py", line 326, in __aexit__
await self.aclose()
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/connection_pool.py", line 313, in aclose
f"The connection pool was closed while {requests_still_in_flight} "
RuntimeError: The connection pool was closed while 2 HTTP requests/responses were still in-flight.

After this change...

$ venv/bin/python ./example.py<Response [301]>

But, if we attempt close the connection pool and then later attempt to read from a connection, we will still get sensible behaviour...

example.py:

importanyioimporthttpcoreasyncdefworker(client, tg):
r=awaitclient.request('HEAD', 'https://google.com')
print(r)
awaitclient.aclose() # Yikes, lookout.asyncdefmain():
asyncwithhttpcore.AsyncConnectionPool() asclient, anyio.create_task_group() astg:
for_inrange(3):
tg.start_soon(worker, client, tg)
if__name__=="__main__":
anyio.run(main)

Output:

$ venv/bin/python ./example.py <Response [301]>
Traceback (most recent call last):
File "./example.py", line 17, in<module>
anyio.run(main)
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_core/_eventloop.py", line 70, in run
return asynclib.run(func, *args, **backend_options)
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 292, in run
returnnative_run(wrapper(), debug=debug)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
returnfuture.result()
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 287, in wrapper
return await func(*args)
File "./example.py", line 13, in main
tg.start_soon(worker, client, tg)
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
raise exceptions[0]
File "/Users/tomchristie/Temp/venv/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 702, in _run_wrapped_task
await coro
File "./example.py", line 6, in worker
r = await client.request('HEAD', 'https://google.com')
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/interfaces.py", line 43, in request
response = await self.handle_async_request(request)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/connection_pool.py", line 253, in handle_async_request
raise exc
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/connection_pool.py", line 237, in handle_async_request
response = await connection.handle_async_request(request)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/connection.py", line 90, in handle_async_request
return await self._connection.handle_async_request(request)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/http11.py", line 112, in handle_async_request
raise exc
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/http11.py", line 91, in handle_async_request
) = await self._receive_response_headers(**kwargs)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/http11.py", line 155, in _receive_response_headers
event = await self._receive_event(timeout=timeout)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_async/http11.py", line 192, in _receive_event
self.READ_NUM_BYTES, timeout=timeout
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/backends/asyncio.py", line 36, inreadreturn b""
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/Users/tomchristie/GitHub/encode/httpcore/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc)
httpcore.ReadError

@lovelydinosaur
lovelydinosaur requested a review from a teamDecember 1, 2022 10:37
@lovelydinosaurlovelydinosaur mentioned this pull request Dec 8, 2022
5 tasks
@Kludex

Copy link
Copy Markdown
Contributor

Closes #631 (Errors will be raised against the connections themselves)

This PR is #631, is that correct?

@lovelydinosaur

lovelydinosaur commented Dec 8, 2022

Copy link
Copy Markdown
ContributorAuthor

This PR is #631, is that correct?

Ooops, updated.
Should have (and now does) read #561.

Comment threadhttpcore/backends/mock.py
@Kludex
Kludex requested a review from zaniebDecember 12, 2022 12:36
@lovelydinosaur
lovelydinosaur merged commit 585b6f3 into masterDec 14, 2022
@lovelydinosaur
lovelydinosaur deleted the drop-runtime-error-on-closing-connection-pool-with-open-connections branch December 14, 2022 15:58
@lovelydinosaur

Copy link
Copy Markdown
ContributorAuthor

Thanks @madkinsz 👍

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Bug List in-flight connections

3 participants

@lovelydinosaur@Kludex@zanieb