HttpConnectionPool improvements

[geoffkizer]

Two related improvements:

(1) Reduce contention on the pool lock. This is done mainly by (a) not calling TrySetResult on the queued waiter under the lock -- instead, do this outside the lock and retry as necessary for canceled requests; (b) avoid doing diagnostic logging under the lock.
(2) Improve handling of failed connection attempts so we don't fail requests unnecessarily. See #60654 for details.

Fixes#60654

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Two related improvements:

(1) Reduce contention on the pool lock. This is done mainly by (a) not calling TrySetResult on the queued waiter under the lock -- instead, do this outside the lock and retry as necessary for canceled requests; (b) avoid doing diagnostic logging under the lock.
(2) Improve handling of failed connection attempts so we don't fail requests unnecessarily. See #60654 for details.

Fixes #60654

Author:	geoffkizer
Assignees:	-
Labels:	area-System.Net.Http
Milestone:	-

[geoffkizer]

/azp run runtime-libraries-coreclr outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[geoffkizer]

@dotnet/ncl Can I get a review?

[geoffkizer]

I pushed fixes for the issues identified above.

[geoffkizer]

/azp run runtime-libraries-coreclr outerloop

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).