Discussion forum for David Beazley

What's the best way to make interfaces with async generators?


If I have some remote resource which continuously produces one or more items and writes them somewhere, what is the best way to deliver the data to the user?

def parse_rawdata(rawdata):
    return rawdata.split(b'SPLITTOKEN')

async def res():
    rawdata = await connection.read()
    data_list = parse_rawdata(rawdata)
    return data_list

What is the best interface for an user to consume the resources? One would be to simply return the whole data list in a loop:

class MyObj:
    def __aiter__(self):
        return self
    async def __anext__(self):
        return await res() 

So the user is expected to do an async loop followed by a normal loop to iterate over the data (not very nice). Another would be async generators, so instead I would do:

async def res():
    rawdata = await connection.read()
    data_list = parse_rawdata(rawdata)
    for ele in data_list:
        yield ele

Unfortunately this is also not very nice, because as of the latest version of curio, the async for loop needs to be wrapped in curio.finalize…


Short answer—having that curio.finalize() in there is annoying. I agree on that. We’re currently discussing options on this here: https://github.com/dabeaz/curio/issues/176

Ultimately, I think the async generator is going to be the best approach. I’m just not sure how it’s going to be packaged up yet.


Curio’s strictness on async generators has been relaxed. Give it a try and see if it works better.


Great! Seems to be working for me. Thanks!