Middleware API

[florimondmanca]

And here I am pitching the idea of a middleware API again… :-)

As I mentioned in https://github.com/encode/httpx/issues/984#issuecomment-633234552, and more recently in #1110 (comment), there is a variety of use cases that don't fall in the "make a custom transport" domain, the "subclass the client without using private API" domain. In general this is transport-agnostic feature that intercepts the request/response cycle, for example...

• Caching: look up the request key in a cache, and optionally return early with a response w/o hitting the transport.
• Throttling: defer the sending of the request until a throttle policy would be satisfied.
• HSTS (?): modify the request to use HTTPS if the requested website is on the HSTS Preload List.

After playing with ideas, this PR is all we would need to support a middleware API that would support the above use cases and, I believe, many more.

Core ideas are…

• Requests are passed through a stack of middleware. The processing stack is:

# Process request...middleware_1 ... middleware_n send_handling_redirects()  # <Ultimately we dispatch to the transport> # Process the response... ... middleware_nmiddleware_1

[florimondmanca]

🤔 -> #1134