Skip to content

asyncio connection_pool: catch CanceledError exception fix connection… - #305

Merged
lovelydinosaur merged 5 commits into
encode:masterfrom
aviramha:fix_cancelederror_pool
Apr 21, 2021
Merged

asyncio connection_pool: catch CanceledError exception fix connection…#305
lovelydinosaur merged 5 commits into
encode:masterfrom
aviramha:fix_cancelederror_pool

Conversation

@aviramha

@aviramhaaviramha commented Apr 19, 2021

Copy link
Copy Markdown
Contributor

… leak in case of cacncellations
closes#149
closeshttps://github.com/encode/httpx/issues/1461

@florimondmanca

Copy link
Copy Markdown
Contributor

Thanks @aviramha

It seems we'd need to fit this into our async backends layer, so that it also works with eg trio (though I'm not sure if we can reproduce the issue there?).

Essentially I think we want the usage to be except self.backend.cancelled_exc_class instead, with that exception being None in the sync case (?).

@aviramha

Copy link
Copy Markdown
ContributorAuthor

I'm not sure what's the functional/design benefit from such behavior - this is the async implementation of connection pool, and catching "CanceledError" in trio won't really do anything anyway so why does it matter?

@lovelydinosaur

Copy link
Copy Markdown
Member

So, Trio's Cancelled exception also inherits from BaseException, yup... https://github.com/python-trio/trio/blob/4e8e19cff8fd5c1b5875d36c77564a6ec021330a/trio/_core/_exceptions.py#L31

I don't think it'd be a terrible idea for us to just catch BaseException here. We are always re-raising the exception.

Either...

exceptNewConnectionRequired:
connection=NoneexceptBaseException: # noqa: PIE786# Note that we catch BaseException here in order to also support# removing connections from the pool on task cancellations.logger.trace("remove from pool connection=%r", connection)
awaitself._remove_from_pool(connection)
raise

Or perhaps...

exceptNewConnectionRequired:
connection=NoneexceptBaseException: # noqa: PIE786try:
logger.trace("remove from pool connection=%r", connection)
awaitself._remove_from_pool(connection)
finally:
raise

@aviramha

Copy link
Copy Markdown
ContributorAuthor

@tomchristie I tend to agree. I updated the PR.

@aviramha
aviramhaforce-pushed the fix_cancelederror_pool branch from f8e9f35 to 12cdeeaCompareApril 20, 2021 14:14
@aviramha
aviramhaforce-pushed the fix_cancelederror_pool branch from 12cdeea to 912c351CompareApril 20, 2021 14:18
Comment threadhttpcore/_sync/connection_pool.py Outdated
Comment threadhttpcore/_sync/connection_pool.py Outdated
Comment threadhttpcore/_async/connection_pool.py
Comment threadhttpcore/_sync/connection_pool.py
@lovelydinosaur
lovelydinosaur merged commit 666cccc into encode:masterApr 21, 2021
@lovelydinosaur

Copy link
Copy Markdown
Member

Thanks @aviramha!

@lovelydinosaurlovelydinosaur mentioned this pull request Apr 21, 2021
@aviramha
aviramha deleted the fix_cancelederror_pool branch April 21, 2021 11:44
@aviramha

aviramha commented Apr 21, 2021

Copy link
Copy Markdown
ContributorAuthor

Thank you for this well maintained library!

Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Pool connection may not be closed correctly in AsyncClient when cancelling the request AsyncConnectionPool leaks connection on CancelledError on 3.8

3 participants

@aviramha@florimondmanca@lovelydinosaur