Summary
If you're writing network-oriented code in Python, you're probably looking at an asynchronous framework of some kind. Let's say you're also writing code that's intended to be used interactively in IPython or Jupyter notebooks.
That's a drag, because async is irrelevant for interactive use, except that you need to sprinkle "await" and "async" keywords in exactly the right places or it doesn't work.
awaitless is an IPython extension that fixes this. It is also the successor to tworoutines, which I now consider fully deprecated.
Tworoutines, Revisited
I've pushed a couple of boulders up this particular hill, and none of them have stayed there.
Trouble is, Python is a great stack for communicating with networked instrumentation, except when asyncio gets involved. And, talking with devices over a network is a picture-perfect use case for asyncio. (The tworoutines page has the longer screed about why we want async code to be usable from a REPL at all.)
My previous attempt was the @tworoutine, which tried to marry async and sync functions under a common wrapper. At the time I wrote it, it was clear tworoutines ran contrary to the direction Python was taking asyncio. Without nest_asyncio, it was fairly brittle, and it has become more brittle over time. The primary developer for nest_asyncio unfortunately passed away in 2024.
There is a long discussion in Issue 22239, continued on GitHub as gh-66435, including representation from Python higher-ups as well as other people writing instrumentation/science software. In other words, it's not just us. (I've posted awaitless there too.) I consider that thread a fairly authoritative record of the current state of affairs, and I consider the tworoutine experiment complete. If you're trying to solve your async Python problems, I recommend against that approach: the odds are stacked against it.
Awaitless narrows the problem. It doesn't try to make async code callable synchronously everywhere. It only fixes the top level of an interactive session, which is the one place where the "await" is never ambiguous.
The Problem
Let's pretend release_kraken is a complicated piece of async machinery. It might take arguments, interacts with the world, and might return a value. Here's a placeholder definition:
>>> async def release_kraken():
... import aiohttp, http
... async with aiohttp.ClientSession() as cs:
... resp = await cs.post('http://httpbin.org/post')
... return resp.status == http.HTTPStatus.OK
Let's say you're calling release_kraken() in an interactive IPython session:
>>> release_kraken()
Out[1]: <coroutine object release_kraken ...>
The kraken is not released. Your function never actually executes. All you get is a "RuntimeWarning: coroutine was never awaited" slap on the wrist. You forgot the "await":
>>> await release_kraken()
Out[1]: True
That's ... better? I mean, it's now "correct" according to asyncio conventions, and we're making use of IPython's autoawait magic, but there is never any ambiguity about a coroutine at the top level of an interactive session. The user wanted to run some code, and forcing them to slavishly type "await" every time is so pedantic it's user-hostile.
Awaitless
Let's do better:
>>> %load_ext awaitless
>>> release_kraken()
Out[1]: <Task finished ... result=True>
What did that do?
- The async code ran to completion, even though we didn't await it (note the 'finished' and 'result').
- The return value is now a Task, not a coroutine. However, both coroutines and Tasks are awaitable, so the switcheroo is (largely) API compatible.
The release_kraken example was picked to focus on an outgoing API call with side effects, to highlight that the coroutine isn't actually executed without the "await" in vanilla IPython. Returned values are equally annoying without awaitless (and just as important in RPC-style code). For example, you can't re-execute a coroutine in vanilla IPython:
>>> c = release_kraken() # returns a coroutine, but it hasn't run yet
>>> await c # the coroutine actually runs here
>>> await c # RuntimeError: cannot reuse already awaited coroutine
With awaitless, this works just fine:
>>> c = release_kraken() # returns a completed Task - the coroutine runs here instead
>>> await c # fine - retrieves the return value from the task
>>> await c # fine - retrieves the *same* return value from the task
Basically, in an interactive session, Tasks are to coroutines as a glass is to orange juice. You probably shouldn't handle the OJ without the glass.
Why?
Yes, "await" is only 6 extra keys to hit, but it's distracting and unnecessary. You could have the same discussion about print()-ing the results in a REPL rather than just displaying them, and IPython cares enough about this to make it configurable. (Try get_ipython().ast_node_interactivity='all' if you're curious.)
I find myself returning to this article every year or so. The author says:
I ain't gonna mock Tcl-scriptable tools no more. I understand what made the authors choose Tcl, and no, it's not just a bad habit. On a level, they chose probably the best thing available today in terms of ease-of-programming/ease-of-use trade-off (assuming they bundle an interactive command shell). If I need to automate something related to those tools, I'll delve into it more happily than previously.
In short: languages designed for interactive use (tcl, bash, etc.) make syntax decisions that are different than languages that are designed for programming. Interactive languages tend to prioritize "shallow" tasks like command invocation at the expense of language composition (functions, classes, modules). IPython is a reasonable compromise, but not when coroutines are involved.
Installing
Something like:
$ pip install awaitless
In IPython or JupyterLab, you can now run
In [1]: %load_ext awaitless
You can make this setting persistent by following the instructions here.
How It Works
Awaitless registers an AST transformer with IPython. Each top-level expression statement or assignment whose value turns out to be a coroutine is wrapped in a Task and awaited before the result is displayed or stored. Code inside function and class definitions is left alone, so the transformation only affects what you type at the prompt, not the libraries you call. It is about 150 lines, and the source is on GitHub.