asyncio.gather is not deprecated and is not going away. But its default
behaviour on failure is the reason services leak tasks, and the standard
library now recommends TaskGroup for new code.
What gather does when something fails
results = await asyncio.gather(fetch_a(), fetch_b(), fetch_c())
fetch_b raises. gather propagates that exception to you immediately — and
fetch_a and fetch_c keep running, detached, with nobody holding their
results.
Trace what that means in a request handler. The request has failed; your
except block has run; you have returned a 500. Meanwhile two database queries
are still in flight against connections checked out of the pool for a request
that no longer exists. They will finish, their results will be discarded, and
if either of them raises, that exception surfaces at garbage-collection time
as “Task exception was never retrieved” with no correlation to anything.
Under load, this is how you exhaust a connection pool without a single line of your code being obviously wrong.
What TaskGroup does instead
async with asyncio.TaskGroup() as group:
a = group.create_task(fetch_a())
b = group.create_task(fetch_b())
The block does not exit until every task in it is done. If one fails with
anything other than CancelledError, the group cancels the remaining tasks
and the enclosing body, waits for them to actually finish unwinding, and then
raises. The invariant is worth stating plainly: when the async with exits,
no task it created is still running. Structured concurrency in one sentence.
Failures come out as an ExceptionGroup — plural, because siblings can fail
concurrently and dropping all but one would be a lie. KeyboardInterrupt and
SystemExit are re-raised directly rather than wrapped, so Ctrl-C still
behaves like Ctrl-C.
You catch it with except*:
try:
async with asyncio.TaskGroup() as group:
...
except* ValueError as group:
for exc in group.exceptions:
log.warning("bad input", exc_info=exc)
Python 3.15 adds TaskGroup.cancel(), which finally allows a non-exceptional
early exit — the “we have enough results, stop” case that previously required
raising a sentinel exception and catching it outside the block.
💡gather(..., return_exceptions=True) returns exceptions in the results list instead of raising. Does that fix the leak?
click to reveal
Yes, and it is the case where gather is still the right tool.
With return_exceptions=True, gather waits for every awaitable and gives
you a list positionally aligned with the inputs, where each element is either a
result or an exception object. Nothing is left running when it returns and
nothing is cancelled. That is genuine partial-results fan-out, and TaskGroup
cannot express it — a TaskGroup’s whole contract is that one failure cancels
the siblings.
So the rule is not “TaskGroup always”. It is:
-
All-or-nothing — one failure makes the whole result meaningless →
TaskGroup. -
Partial results — you want whatever succeeded and a record of what did
not →
gather(..., return_exceptions=True), or explicit tasks plusasyncio.wait. -
Bare
gather— almost never. It combines “fail fast” with “leak the siblings”, which is the one combination nobody wants.
The typing argument, which is more practical than it sounds
typeshed gives gather overloads for exactly six positional awaitables, so
it can give you a precisely-typed tuple back. Pass a seventh, or splat a list,
and you fall off the end of the overloads and the return type collapses to
list[Any].
That is not a cosmetic problem. Any is contagious: every element you pull out
of that list is Any, everything you build from those elements is Any, and
--strict will not say a word — the syllabus item on what --strict misses
covers exactly this laundering. A fan-out over a list of keys is the common
case, so in practice the typed fan-out is the one that keeps tasks in a
dict[str, Task[T]] and reads results off the tasks.
💡A TaskGroup body creates three tasks and then the body itself raises before the block ends. What happens to the three tasks? click to reveal
They are cancelled, awaited to completion, and their outcomes are folded in with the body’s exception.
That is the part people miss: the async with block’s own code is inside the
scope, not outside it. If the body raises, the group cancels every task it
created, waits for each to finish unwinding — which means their finally
blocks and except CancelledError handlers really do run — and then raises.
The body’s exception and any non-cancellation task failures come out together
in an ExceptionGroup.
So there is no path out of the block that leaves work running, including
return, break, an exception, and the enclosing task being cancelled from
outside. Compare gather, where the failure path leaves siblings detached with
nobody holding them — that is the whole difference in one sentence.