Skip to content

← Concurrency with Threads and asyncio step 4 of 56

Easy Primitives

Collect Results and Errors from a Thread Pool

You are writing a health check for a list of websites. Some sites answer, some are down, and one bad site must not stop you checking the rest. Run the checks on a thread pool and sort the outcomes into successes and failures.

FakeChecker.check(url) stands in for the real check. It waits a moment, then returns a status code or raises an exception:

  • A URL containing "down" raises ConnectionError.
  • A URL containing "timeout" raises TimeoutError.
  • A URL containing "missing" returns 404. Every other URL returns 200.
check_all(FakeChecker(), ["ok-1", "down-2", "missing-3"], max_workers=3)
# ({"ok-1": 200, "missing-3": 404},
#  {"down-2": "ConnectionError: down-2 is unreachable"})

What to write

Fill in check_all(checker, urls, max_workers). It returns two dictionaries, (ok, failed):

  • Submit checker.check(url) once for each URL to a ThreadPoolExecutor with max_workers workers.
  • If a check returns, put its status code in ok under its URL. A 404 is a result like any other, so it goes in ok.
  • If a check raises, put "<ExceptionName>: <message>" in failed under its URL. For example, ConnectionError("down-2 is unreachable") becomes "ConnectionError: down-2 is unreachable".
  • A failure must not stop the other checks. Every URL ends up in exactly one of the two dictionaries.

The URLs are all different, urls can be empty, and max_workers is at least 1.

What the driver checks

Keep FakeChecker and solve as they are. solve returns your two dictionaries as ok and failed, plus:

  • calls: how many times check ran. It must equal the number of URLs.
  • overlapped: True if two or more checks ran at the same time. With one URL or one worker, the expected value is False.