Skip to content

← Concurrency with Threads and asyncio step 21 of 56

Medium Primitives

Accept Any Fetcher You Can Await

You are writing collect, which takes a fetcher and a list of keys and returns a dictionary of results. Three fetchers already exist, and each one's fetch returns something different:

Class fetch returns Awaited value for key "a"
AsyncFetcher a coroutine (it uses async def) b"a!"
FutureFetcher a future whose result is already set b"A"
TaskFetcher a task it has just started b"task:a"

The starter's Fetcher protocol uses async def fetch, so mypy rejects the last two in _REGISTRY. Fix the protocol, then write collect.

await collect(AsyncFetcher(), ["b", "a", "b"])
# {"b": b"b!", "a": b"a!"}, and fetch ran for "b" and then "a"

What to write

Edit only Fetcher and collect. Keep the three fetchers and the driver as they are.

  • Fetcher[T] is a generic protocol. Fetcher[bytes] means a fetcher whose results are bytes. Change its method to a plain def fetch(self, key: str) -> Awaitable[T]: ..., so all three classes in _REGISTRY fit it.
  • Write async def collect[T](fetcher: Fetcher[T], keys: Sequence[str]) -> dict[str, T].
  • Go through keys in order, one key at a time. For each key you have not seen yet, call fetcher.fetch(key) once and await the result before you move on.
  • Skip a repeated key before you call fetch. Calling it again would start a second task in TaskFetcher.
  • Store the awaited value under its key, not the coroutine, task or future. The dictionary's keys end up in first-seen order.
  • If fetch or the await raises, let the exception propagate. Don't retry.

Edge cases

  • An empty keys list returns {} without calling fetch.
  • Skip repeats by key, not by value. FutureFetcher gives b"A" for both "a" and "A", and both keys belong in the result.

What the driver reports

  • payloads is the dictionary collect returned.
  • calls lists every key passed to fetch, in call order.
  • order lists the keys of payloads in order.

Both lists must be in first-seen order, with one entry per distinct key. For ["b", "a", "b"], calls and order are both ["b", "a"].