We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Concurrency Without Regret step 17 of 38
fetch_all: structured fan-out with TaskGroup
Write both halves of fan-out: the all-or-nothing one and the partial-results one.
async def fetch_all[T](
fetch: Callable[[str], Awaitable[T]], keys: Sequence[str]
) -> dict[str, T]
async def fetch_all_lenient[T](
fetch: Callable[[str], Awaitable[T]], keys: Sequence[str]
) -> tuple[dict[str, T], dict[str, BaseException]]
Both fetch each distinct key once, and both key their results by key, in first-seen order.
-
fetch_alluses anasyncio.TaskGroup. The first failure cancels the siblings and the call raises anExceptionGroup. No task may still be running when the call returns or raises. -
fetch_all_lenientattempts every key to completion. One failure cancels nothing; you get back what succeeded and what did not.
Why both exist
Bare asyncio.gather gives you the one combination nobody wants: the first
exception propagates and the siblings keep running, detached. In a request
handler that means the request has failed, you have returned a 500, and two
queries are still in flight on connections checked out for a request that no
longer exists. Under load that is how a connection pool is exhausted without a
single obviously wrong line of code.
TaskGroup‘s contract is one sentence: when the async with exits — by
return, by break, by an exception in the body, or by the enclosing task
being cancelled — no task it created is still running.
gather(..., return_exceptions=True) is still correct for the lenient case,
and is not deprecated. Note though that typeshed only has overloads for six
positional awaitables; splat a list and the return type collapses to
list[Any], silently erasing the element type through everything downstream.
Keeping the tasks in a dict[str, Task[T]] and reading results off them keeps
the types.
What the report proves
-
values— the awaited payloads, asbytes, keyed and ordered by first appearance. -
started— which fetches began, in order. -
cancelled— which fetches observed aCancelledError. On the strict path with a failing key, the slow siblings must appear here. On the lenient path this must stay empty. -
outcome/group_size/group_types— the strict path raises anExceptionGroupcontaining exactly the real failures, not a bareValueErrorand not a group with the cancellations folded in. -
errors— the lenient path’s failure map. -
pending— tasks still alive when the driver looks. Must be0on every path. This is the assertiongatherfails.
Where the type system earns its keep
Both functions are generic in the fetched type, so a Callable[[str], Awaitable[bytes]] gives you dict[str, bytes] and not dict[str, Any].
TaskGroup.create_task wants a Coroutine, not an Awaitable — so a
Callable[[str], Awaitable[T]] cannot be handed to it directly, and the
one-line async def wrapper that fixes it is the type system telling you
something true about laziness.
Loading visualization…
Stuck?
Python reference solution
Sign in to attempt this problem and reveal the reference solution.