ABC for Futures

[1st1]

I'm experimenting with implementing an asyncio-compatible event loop on top of libuv with Cython. It would be great if there is a way to provide a custom Future implementation. I was thinking about adding loop.create_future() (instead of Future(loop=loop)) API method, as well as an ABC, so that functions like ensure_future would recognize custom Futures. @gvanrossum @Haypo what do you think about this?

[gvanrossum]

Hm... Why can't your new loop reuse the existing Future class? Or subclass
it?

On Sun, Nov 1, 2015 at 4:45 PM, Yury Selivanov notifications@github.com
wrote:

I'm experimenting with implementing an asyncio-compatible event loop on
top of libuv with Cython. It would be great if there is a way to provide a
custom Future implementation. I was thinking about adding
loop.create_future() (instead of Future(loop=loop)) API method, as well
as an ABC, so that functions like ensure_future would recognize custom
Futures. @gvanrossum https://github.com/gvanrossum @Haypo
https://github.com/haypo what do you think about this?

—
Reply to this email directly or view it on GitHub
#282.

--Guido van Rossum (python.org/~guido)

[1st1]

I can reuse the existing Future (but I can't derive extension types from Python classes in Cython). I just wanted to make a little bit faster Future, and, especially, Task classes.

Maybe I'm trying to micro-optimize here.

[vstinner]

Hack asyncio in a local branch, come with a microbenchmark and then we can discuss ;-) Once I implemented a free list for Task. The gain was null, so I gave up.

[1st1]

@Haypo Yeah, I'm closing this issue for now. I'll come back with a microbenchmark if there is any difference.

[1st1]

@gvanrossum @Haypo @asvetlov So now I have benchmarks ;)

Long story short, here is an implementation of an asyncio-compatible event loop on top of libuv: https://github.com/1st1/uvloop. It's an early preview, however, at this point enough is implemented to run various benchmarks. I think that I got most of the low-level implementation right, now I need to add more tests and implement the missing APIs.

For benchmarks I used the ones that David Beazley used for his curio project.

Echo server using low-level loop.sock_recv / loop.sock_sendall

Results, Python 3.5, best of 3, 600,000 ~1kb messages sent in 3 parallel processes to a single server process:

1. asyncio with built-in event loop

   600000 in 43.61588096618652
   13756.457205694267 requests/sec
   

2. asyncio with uvloop

   600000 in 17.48739981651306
   34310.418146523414 requests/sec
   

3. gevent (Python 2.7)

   600000 in 15.572859048843384
   38528.57064448694 requests/sec
   

4. curio

   600000 in 21.155117988586426
   28361.93115650364 requests/sec
   

Echo server using asyncio streams

Results, Python 3.5, best of 3, 600,000 ~1kb messages sent in 3 parallel processes to a single server process:

1. asyncio with built-in event loop

   600000 in 28.544986963272095
   21019.452584511615 requests/sec
   

2. asyncio with uvloop

   600000 in 14.964998960494995
   40093.55440544273 requests/sec
   

Analysis

Bottom line, asyncio with uvloop is about ~1.85-2.5x faster.

I was initially surprised to see that the streams server was so fast, since there is quite a bit of Python code in asyncio/streams.py. The explanation is that in uvloop, transports are implemented directly on top of libuv streaming abstractions. All buffering and low-level IO is managed by libuv directly and efficiently.

Future objects in C

The above results were produced with unmodified asyncio (stock Future, stock Task, stock streams.py).

In order to see if I can further improve the performance, I implemented Futures in pure C: uvloop/future.c. In microbenchmarks, C version is 2x faster than asyncio/futures.py.

Here's the result of running sock_recv / sock_sendall benchmark with C Future:

600000 in 12.447252035140991
48203.41054443859 requests/sec

Another ~25% improvement (3.5x faster than vanilla asyncio, 1.25x faster than gevent), absolute win! I think it would make sense to implement this as well. Below is a list of modifications to asyncio that we would need:

1. Future has to be an ABCMeta class, so that existing isinstance checks will continue working with new futures.
2. isinstance checks are very slow, and are visible when profiling, especially in Task._step. I'm thinking about something like loop.is_future method, that can be implemented very efficiently. Alternatively, we can just use duck-typing (in Task._step) by substituting isinstance(result, Future) with hasattr(result, 'add_done_callbacks')
3. In order to make it possible to switch all code to new futures, we'll need loop.create_future (similar to the existing loop.create_task).
4. The existing Future class has some methods that shouldn't be methods at all, for instance _set_result_unless_cancelled or _copy_state. Those only use Future's public API, and can easily be factored out into separate helper functions.
5. Future._loop and Future._blocking have to become public API (get_loop(), was_yielded()?).

[gvanrossum]

Exciting -- but can you find some collaborators to help with this? I can't take this on. :-(

[1st1]

@gvanrossum To be clear, I'm not proposing to merge uvloop into asyncio, it should stay as a separate project for a while, before we can even consider that. I want to do some modifications to the asyncio.Future - mainly inheriting it from ABCMeta, stripping some of its undocumented APIs, and adding loop.create_future() method. If you and @Haypo can give me a greenlight to do that, I'd be happy to create a PR.

[gvanrossum]

Do the modifications also benefit plain vanilla asyncio? By how much? I am hesitant to change any public APIs; in particular requiring people to call loop.create_future() seems onerous. Would it be possible to gain a similar speedup just by rewriting some of asyncio.Future's internals, without any public API changes?

[1st1]

I am hesitant to change any public APIs; in particular requiring people to
call loop.create_future() seems onerous.

Calling asyncio.Future(loop=loop) isn't much different from loop.create_future(). The latter form is less error prone, since the future will always be attached to the correct loop. I, myself, always pass the loop argument manually. And, by the way, nothing will prevent people from using asyncio.Future, as loops must work with any compatible implementation.

In my experience (and I may be wrong here), people don't create futures by hand often. Instead they use asyncio through frameworks like aiohttp, which simply expose some awaitables.

Would it be possible to gain a similar speedup just by rewriting some of
asyncio.Future's internals, without any public API changes?

It's a bit tricky. Essentially, my C implementation of the Future class is very similar to the Python one. Here are some reasons why it's faster:

1. Since it's a C object, there is no __dict__ lookups. And being implemented in C, we don't have opcodes/ceval overhead.
2. __iter__ and __await__ return the future object itself, and the future implements a __next__ slot in a very straightforward way. Python version, in contrast, creates a generator on each call to __await__. In Python it's a better strategy, but in C my approach is much faster.

Speaking about new APIs, I don't think we need much. Many of the ones I listed above are very nice to have, but let me iterate through them and highlight the most important ones:

1. [important] The main one is making asyncio.Future an ABCMeta. This will allow me to have a custom Future implementation and all existing asyncio code (ensure_future etc) will accept it.
2. [important] Future._set_result_unless_cancelled and Future._copy_state need to be factored out to standalone functions. This shouldn't be a problem - they aren't part of any public API, they themselves use only public API, and they are used in just a few places in asyncio. To preserve absolute backwards compatibility we can keep the methods, but modify the code that uses them to use standalone functions.
3. [very nice to have] loop.create_future() method. I think it'd be quite handy to use. It will improve performance in any code that uses it and is running with uvloop or any other optimized event loop.
4. [optional] Make public APIs for Future._blocking and Future._loop. These I can easily implement in C as getters/setters, so this is really optional.
5. [optional] I'd want to slightly tweak Task._step and Task._wakeup. The former will use hasattr(result, 'add_done_callback') instead of isinstance(result, Future) (duck typing). And it would be nice to change the latter to call self._step slightly different (I can explain in the patch why). This change will also slightly improve performance of asyncio.Future.

I'd be very willing to contribute my Future implementation to asyncio and CPython. But I'm not sure what's the timeframe here is going to be. Everything except loop.create_future is very lightweight in terms of API changes, and loop.create_future is something that I genially believe would be a great addition regardless of its performance benefits.

[vstinner]

Hum, I'm not sure that we want an ABC here. Yury proposes a C accelerator module for asyncio. I really like this idea, I had the same idea many times (I tried to implement a free list for the Task class, but I didn't see any difference in performance). I know that writing the same code with the Python C API makes the code faster (I don't understand why, it's just a fact :-)).

IMHO we need an _asyncio module which implements some functions and classes of asyncio, but keep the Python implementation for other Python VMs (PyPy & cie). And then use the classic "try: from _asyncio import Future except ImportError: Future = PyFuture".

"Calling asyncio.Future(loop=loop) isn't much different from loop.create_future(). The latter form is less error prone, since the future will always be attached to the correct loop."

We can consider this as a different issue. For me, it's a an API enhancement to make the loop "parameter" mandatory (and it might be a little bit faster, no need to get asyncio.get_event_loop()). The drawback is that you have to pass loop to a lot of functions. I don't know if users like the implicit loop parameter. Maybe passing explicitly the loop parameter matters more when you have multiple event loops running in different threads? (but asyncio.get_event_loop() handles this use case thanks to thread local storage).

I'm ok to add loop.create_future() and try to use it in asyncio core, but I'm not sure yet that we really must to replace all Future(loop=loop) with loop.create_future(). As the addition of loop.create_task(), it also creates a problem with backward compatibility. To write code working with asyncio of Python 3.4.0 stdlib, create_task() and create_future() cannot be used... as await/async keywords...

[asvetlov]

Very exciting results!

1 Making asyncio.Future an ABCMeta.

Agree.

2 Future._set_result_unless_cancelled and Future._copy_state need to be factored out to standalone functions.

I see no problems with modifying private part if asyncio on our own.

3 loop.create_future() method.

Totally support the idea. loop.create_future() looks like loop.create_task() and has similar motivation.

4 Make public APIs for Future._blocking and Future._loop.

I doubt we need it public, especially _blocking attribute. There is nothing bad with public .loop though.

5 I'd want to slightly tweak Task._step and Task._wakeup. The former will use hasattr(result, 'add_done_callback') instead of isinstance(result, Future) (duck typing).

I don't like duck typing in general but in this case we can live with it if the change improve performance significantly.

[asvetlov]

@Haypo I think Future implementation may be coupled event loop, so having the single _asyncio.Future may not satisfy all requirements. loop.create_future() looks much more clean and native.

About explicit/implicit loop. I've found the former extremely important, all my public libraries and private code use explicit loop only. It helps with code and tests isolation very well and don't bring too much work with passing loop parameter everywhere.
Mixing loops (e.g. explicit and implicit) leads to hard-to-fix errors. I consider libraries that doesn't accept explicit loop as not ready to be used.

Inexperienced users likes implicit loop, yes -- until they shoot own leg by the technique. After that they switches to explicit one. I saw it several times.

About backward compatibility problems. .create_future() is not mandatory, developers still may use Future(loop=loop) call.

Library authors may adopt their code to use loop.create_future() if available. That's easy and don't require a lot of modifications. Say, aiohttp has 6 places with explicit future creation only, the same is true for all my other libraries.

Yes, aiohttp like any other asyncio library creates a lot of future instances in test suite but as Future() is still supported the test suite doesn't require any change.

I think 25% speedup merits adding new optional method to event loop. I definitely appreciate changes which makes aiohttp-based applications several times faster.