Skip to content

← Error Handling with Exceptions, Retries and Warnings step 9 of 20

Medium Primitives

Fix a Batch Runner That Catches Too Much

A batch runner calls independent tasks one after another. If a task fails, the runner records the error and starts the next task. If you press Ctrl-C, the whole batch stops. The starter's run_all catches BaseException, so Ctrl-C only skips one task. Fix it, and write solve to report what the runner did.

tasks:  [{}, {"raises": "ValueError", "message": "row 1"}, {}]
report: {"outcome": "completed", "abort_type": None, "started": [0, 1, 2],
         "failures": [{"index": 1, "type": "ValueError", "message": "row 1"}]}

tasks:  [{}, {"raises": "KeyboardInterrupt"}, {}]
report: {"outcome": "aborted", "abort_type": "KeyboardInterrupt",
         "started": [0, 1], "failures": []}

Fix run_all

run_all(tasks) calls each task once, in order. Keep its return type, list[tuple[int, Exception]].

  • If a task raises an Exception, append (index, exc) and go on. index counts from zero. Append the object you caught, so its notes and attributes survive.
  • If a task raises a BaseException that is not an Exception, such as KeyboardInterrupt, SystemExit or GeneratorExit, let that same object escape at once. Do not call later tasks, and do not return the failures so far.
  • A task that succeeds adds nothing. An empty list of tasks returns [].

The starter also fails mypy, because a BaseException does not fit in that list. Catching Exception fixes the type error and the bug together. Widening the return type would only silence mypy.

Write solve

solve(tasks) gets a list of specs, each a dictionary of strings. Turn each spec into a function that takes no arguments:

  • The function first appends its own index to a shared started list.
  • Without a "raises" key, the function returns and ignores any "message".
  • With "raises", it raises RAISABLE[name](message). name is the "raises" value, and message is the spec's "message", or "" if there is none.

Each function must keep its own index and spec. A function defined directly in the loop would see the loop's last values when it runs.

Call run_all with the functions, and catch BaseException around that call to build the aborted report.

What solve returns

Key Value
"outcome" "completed" if run_all returned, "aborted" if an exception escaped
"abort_type" The escaped exception's class name, or None
"started" The indices of the tasks that ran, in order
"failures" One dictionary with "index", "type" and "message" per recorded failure, or [] after an abort

Use type(exc).__name__ for "type" and str(exc) for "message". str(KeyError("head")) is "'head'", with quotes.