Skip to content

← Failure by Design step 8 of 18

Hard Primitives

ExceptionGroup, split() and except*

Concurrency changed the shape of failure. When ten tasks run together and four of them fail differently, the pre-3.11 answer was to pick one and lose the other three. ExceptionGroup (PEP 654) exists so that a failure can be a tree rather than a single value.

Two classes, and the difference is load-bearing:

  • ExceptionGroup may only contain Exception members, and is itself an Exception — so except Exception catches it.
  • BaseExceptionGroup may contain anything, and is deliberately not an Exception when it holds a non-Exception member. That is precisely what lets asyncio.CancelledError travel inside a group without a blanket except Exception swallowing a cancellation.

The constructor picks for you: BaseExceptionGroup(msg, excs) returns an ExceptionGroup when every member is an Exception, and a plain BaseExceptionGroup otherwise. You will see that in your own output.

The programmatic API

except* is the syntax everyone learns first, but the interesting work happens through three methods:

  • eg.split(condition) returns (matching, rest), each an ExceptionGroup-shaped tree of the same shape or None. condition is an exception class, a tuple of classes, or a predicate.
  • eg.subgroup(condition) is split(...)[0].
  • eg.derive(excs) builds a new group of the same class, and is the hook you override in a custom group subclass.

Splitting recurses. A nested group whose matching members are two levels down comes back with the nesting preserved, and empty branches pruned. message and __notes__ are copied onto every derived group, so context you attached with add_note() survives the split — which is exactly why add_note() is worth using in the first place.

What to build

Three functions.

build(spec) turns a nested mapping into a real exception tree. A spec with an "items" list is a group: build each child and return BaseExceptionGroup(message, children). Otherwise it is a leaf: instantiate LEAF_TYPES[spec["exc"]](message). In both cases, if the spec has a "note" string, attach it with add_note().

partition_errors(eg, *, retryable) does the actual work, in two steps and in this order:

  1. eg.split(Exception). The rest — anything that is not an Exception — is not yours to classify. raise it. Swallowing a cancellation because it happened to be travelling next to a retryable error is one of the nastiest bugs in async Python.
  2. Split the remaining Exception half on retryable and return (matching, rest) directly. Either half may be None; that is the API telling you “nothing matched”, and None is the right answer, not an empty group.

describe(exc) renders a tree as plain data: a group becomes {"kind": "group", "type": <class name>, "message": <group message>, "notes": [...], "items": [...]}, and a leaf becomes {"kind": "leaf", "type": ..., "message": str(exc), "notes": [...]}. notes is [] when the exception has no __notes__ attribute at all — note that __notes__ does not exist until add_note() is called once.

The probe

def solve(tree: Mapping[str, object], retryable: Sequence[str]) -> dict[str, object]:

Build the tree, look the retryable class names up in RETRYABLE_TYPES, and partition. Return either

{"outcome": "partitioned", "retryable": <described or None>, "terminal": <described or None>}

or, when a non-Exception escaped,

{"outcome": "escaped", "escaped": <described>}

You can catch a group with an ordinary except BaseExceptionGroup as eg — it is except* BaseExceptionGroup that is forbidden, not this.

Why there are two lookup tables

LEAF_TYPES is Mapping[str, type[BaseException]]; RETRYABLE_TYPES is Mapping[str, type[Exception]] and is deliberately missing Cancelled and KeyboardInterrupt. That is not tidiness. partition_errors declares retryable: tuple[type[Exception], ...], so a type[BaseException] will not type-check as a member — the type system physically prevents you from marking a cancellation as retryable. This is what “make illegal states unrepresentable” looks like in ninety seconds of annotation work.

The except* gotcha to carry away

except* ValueError as eg binds an ExceptionGroup[ValueError], not a ValueError. Every clause runs that has a non-empty subgroup, so two clauses can both fire on one group; anything unmatched propagates automatically. You may not mix except and except* on one try, you may not write except* ExceptionGroup, there is no bare except*:, and continue, break and return are all syntax errors inside an except* clause.

Retrying the retryable half is the subject of the retry problem in this track; here, get the partition exactly right.

Loading visualization…