Discussion forum for David Beazley

Should a decorator of an async function also be an async function?


So, here’s a little thing I’ve been thinking about lately. Here are two different decorator functions::

from functools import wraps

def decorate1(func):
    def wrapper(*args, **kwargs):
        print('Calling', func.__name__)
        return func(*args, **kwargs)
    return wrapper

def decorate2(func):
    async def wrapper(*args, **kwargs):
  	    print('Calling', func.__name__)
        return await func(*args, **kwargs)
    return wrapper

Let’s apply each decorator to an asynchronous function:

async def spam1():

async def spam2():

Now, let’s call the two functions:

>>> c1 = spam1()
Calling spam1
>>> c1
<coroutine object spam1 at 0x105516e08>

>>> c2 = spam2()
>>> c2
<coroutine object spam2 at 0x109461e08>

In both cases, you get a coroutine object. However, the code in the first decorator fires right away. The code in the 2nd decorator does not. You won’t see that code execute until you hand the coroutine off to another library such as Curio or asyncio. For example:

>>> from asyncio import get_event_loop
>>> loop = get_event_loop()
>>> loop.run_until_complete(c2)
Calling spam2

This begs a somewhat interesting design question. If you’re wrapping an asynchronous function, when should that code execute? Should it execute right away at function application time (decorate1)? Or should it be deferred until the wrapped async function actually starts to run (decorate2)?

I’m leaning towards the latter although it all seems rather tricky. Would love to get your thoughts…


my initial impulse was “setup vs runtime”, assuming the latter would be the predominant one.

will think about this a bit. Not sure what a setup decorator on an async would practically be - async partials? logger or translation, protocol settings? etc.


Even something like logging is surprisingly tricky. In the case of decorator1, a logging message would potentially be issued much earlier than when the wrapped coroutine actually executed. So, the log file would be highly misleading. You’d see the message issued at some point and think that the function ran, not knowing that it got deferred to some later point in time.


Well probably one should care if the decorator either

a) additionally awaits something besides the wrapped coroutines, or

b) has side effects, which happen before calling or awaiting.

Or both. I’m not sure I have a preference at this point but maybe the question to ask is “what would we want if this was just normal ‘synchronous’ code?” I think we’d want the wrapper to execute along with the wrapped.


For something like the async_thread await(...) function, decorate1 will execute in the worker thread while decorate2 will execute in the curio thread. I guess this somehow speaks to the discussion in https://github.com/dabeaz/curio/issues/122.

Another difference is whether you get to see the wrapper function inside tracebacks. I’ve been tending towards the decorator2 style because of this, out of a vague general idea that more-detailed tracebacks are probably a good default, but I suppose there are arguments either way.

It also affects introspection: inspect.iscoroutinefunction gives very different answers for those two decorated functions. From a quick grep of the curio source code, it looks like this could cause confusion for abide and AsyncABCMeta, which both use iscoroutinefunction. (Maybe you need a version of iscoroutinefunction that walks the .__wrapped__ chain?)

I did run into one case where the difference really mattered. It’s extremely weird and doesn’t generalize at all, but it might amuse folks :-). I have a decorator whose job is just to set a local variable with a magic name and then call the wrapped function; the local variable is used to “mark” that callstack, so when a signal handler fires it can walk the stack and look for the setting of this variable. I got extremely confused when I realized that my code was running perfectly except that sometimes the decorator worked and sometimes it didn’t, until I realized that decorate1-style wrappers actually work perfectly on async functions except that they disappear from the call stack, and thus become invisible to the stack walking code…


Huh. In the above post, the forum’s markdown parser appears to be stripping spaces around backticks in some cases but not others. What a weird bug.


Wow, I hadn’t even thought about the call-stack issue associated with this. That makes it even more insane.

Maybe it’s just me, but I feel like coding with async functions is a whole different realm of programming. I find myself having to relearn just about everything–or least finding that any existing assumption I might have had about something is now upended in surprising ways.

With respect to https://github.com/dabeaz/curio/issues/122, I’m still thinking about that. I’ve been working on some new things where changing it would probably make a lot of sense.


FAQ: Does Python support the tail-call optimization?


def factorial(n):
    async def helper(n, accumulator):
        return accumulator if n == 0 else helper(n - 1, n * accumulator)
    result = helper(n, 1)
    while not isinstance(result, int):
        result = curio.run(result)
    return result


I had this doubt too and that is why I landed in this page. In the first case(decorate1) you are replacing a coroutine with a method. In the second case(decorate2) you are replacing a coroutine with another coroutine.

The second one is more appropriate. In case someone decides to use your code the developer would expect the second case.


By the way, the things get “trickier” if the decorator wants to do things after the calls of the decorated function

for example, if I have

def decorator(fun):
    def wrapper(*args, **kwargs):
            return func(*args, **kwargs)
    return wrapper

and I use this decorator to decorate an async function, “before” AND “after” lines appears before the coroutine execution…

btw: does functools.wraps return an ‘async’ function decorator if you pass an async function to decorate? It’s indifferent? or it’s necessary to create a new “functools.async_wraps”?