Skip to content

← Concurrency with Threads and asyncio step 13 of 56

Medium Primitives

Bring Thread Errors Back to the Caller

A plain thread drops its function's result, and Python only prints its exception. Write the helper that brings both back. run_workers runs each function in its own thread and waits for all of them, up to a deadline. Then it returns their results, or raises the first error in list order, as if the functions had run in the caller's own thread.

run_workers([lambda: 1, lambda: 2], timeout=5.0)       # [1, 2]
run_workers([lambda: 1, lambda: 1 // 0], timeout=5.0)  # raises ZeroDivisionError
run_workers([lambda: time.sleep(60)], timeout=0.5)     # raises TimeoutError

What to write

def run_workers[T](fns: Sequence[Callable[[], T]], timeout: float) -> list[T]:
  • Run each function in fns in its own new thread. Use non-daemon threads, which is the default.
  • Start every thread first. Then set one deadline, timeout seconds from now, using time.monotonic(). Pass each join only the time left until that deadline, so the whole wait lasts about timeout seconds in total.
  • After the joins, if any thread is still alive, raise TimeoutError. A timeout takes priority over a failure, because some results are still missing.
  • Otherwise, if any function raised, raise the exception from the lowest index that failed. Raise the original exception object, so its traceback still points at the line that failed. Catch BaseException in the thread, so that every kind of exception reaches the caller.
  • Otherwise, return the results in the same order as fns.

Edge cases

  • An empty fns returns [].
  • With a timeout of zero or less, run_workers checks the threads right away. Threads that have already finished still count as finished.
  • Raising TimeoutError leaves the slow threads running, because Python can't kill a thread. The driver stops and joins them after you return. In real code, the caller owns that cleanup.

What the driver checks

Keep solve as it is. It builds one function for each entry in kinds. An "ok" function sleeps briefly and returns a number. Functions later in the list sleep less, so they tend to finish first. A "value_error" or "runtime_error" function raises, and a "hang" function waits up to 5 seconds for the driver's stop signal. The report has these keys:

  • outcome is "ok" or the name of the exception your function raised, and message is its text. The driver does not check the message of a TimeoutError, so yours can say anything.
  • values is the list you returned.
  • For a worker's error, traceback_ok is True when the traceback still includes _explode, the function that raised. Raising a new exception with the same message fails this check.
  • any_daemon and on_main are True if any function ran on a daemon thread or on the main thread. Both must be False.
  • threads counts the distinct threads that ran a function. It must equal len(fns).