Skip to content
← All articles

TaskGroup Supersedes gather

gather's failure mode leaves siblings running detached. TaskGroup guarantees that nothing it started outlives the block — and gather is still right for partial results.

asyncio.gather is not deprecated and is not going away. But its default behaviour on failure is the reason services leak tasks, and the standard library now recommends TaskGroup for new code.

What gather does when something fails

results = await asyncio.gather(fetch_a(), fetch_b(), fetch_c())

fetch_b raises. gather propagates that exception to you immediately — and fetch_a and fetch_c keep running, detached, with nobody holding their results.

Trace what that means in a request handler. The request has failed; your except block has run; you have returned a 500. Meanwhile two database queries are still in flight against connections checked out of the pool for a request that no longer exists. They will finish, their results will be discarded, and if either of them raises, that exception surfaces at garbage-collection time as “Task exception was never retrieved” with no correlation to anything.

Under load, this is how you exhaust a connection pool without a single line of your code being obviously wrong.

What TaskGroup does instead

async with asyncio.TaskGroup() as group:
    a = group.create_task(fetch_a())
    b = group.create_task(fetch_b())

The block does not exit until every task in it is done. If one fails with anything other than CancelledError, the group cancels the remaining tasks and the enclosing body, waits for them to actually finish unwinding, and then raises. The invariant is worth stating plainly: when the async with exits, no task it created is still running. Structured concurrency in one sentence.

Failures come out as an ExceptionGroup — plural, because siblings can fail concurrently and dropping all but one would be a lie. KeyboardInterrupt and SystemExit are re-raised directly rather than wrapped, so Ctrl-C still behaves like Ctrl-C.

You catch it with except*:

try:
    async with asyncio.TaskGroup() as group:
        ...
except* ValueError as group:
    for exc in group.exceptions:
        log.warning("bad input", exc_info=exc)

Python 3.15 adds TaskGroup.cancel(), which finally allows a non-exceptional early exit — the “we have enough results, stop” case that previously required raising a sentinel exception and catching it outside the block.

💡gather(..., return_exceptions=True) returns exceptions in the results list instead of raising. Does that fix the leak? click to reveal

Yes, and it is the case where gather is still the right tool.

With return_exceptions=True, gather waits for every awaitable and gives you a list positionally aligned with the inputs, where each element is either a result or an exception object. Nothing is left running when it returns and nothing is cancelled. That is genuine partial-results fan-out, and TaskGroup cannot express it — a TaskGroup’s whole contract is that one failure cancels the siblings.

So the rule is not “TaskGroup always”. It is:

  • All-or-nothing — one failure makes the whole result meaningless → TaskGroup.
  • Partial results — you want whatever succeeded and a record of what did not → gather(..., return_exceptions=True), or explicit tasks plus asyncio.wait.
  • Bare gather — almost never. It combines “fail fast” with “leak the siblings”, which is the one combination nobody wants.

The typing argument, which is more practical than it sounds

typeshed gives gather overloads for exactly six positional awaitables, so it can give you a precisely-typed tuple back. Pass a seventh, or splat a list, and you fall off the end of the overloads and the return type collapses to list[Any].

That is not a cosmetic problem. Any is contagious: every element you pull out of that list is Any, everything you build from those elements is Any, and --strict will not say a word — the syllabus item on what --strict misses covers exactly this laundering. A fan-out over a list of keys is the common case, so in practice the typed fan-out is the one that keeps tasks in a dict[str, Task[T]] and reads results off the tasks.

💡A TaskGroup body creates three tasks and then the body itself raises before the block ends. What happens to the three tasks? click to reveal

They are cancelled, awaited to completion, and their outcomes are folded in with the body’s exception.

That is the part people miss: the async with block’s own code is inside the scope, not outside it. If the body raises, the group cancels every task it created, waits for each to finish unwinding — which means their finally blocks and except CancelledError handlers really do run — and then raises. The body’s exception and any non-cancellation task failures come out together in an ExceptionGroup.

So there is no path out of the block that leaves work running, including return, break, an exception, and the enclosing task being cancelled from outside. Compare gather, where the failure path leaves siblings detached with nobody holding them — that is the whole difference in one sentence.