Skip to content

← Concurrency Without Regret step 19 of 38

Hard Primitives

with_cleanup: bounded cleanup on every exit path

Cleanup that runs on every exit path, costs a bounded amount, and never lies about what happened.

async def with_cleanup[T](
    body: Callable[[], Awaitable[T]],
    cleanup: Callable[[], Awaitable[None]],
    *,
    cleanup_timeout: float,
) -> T

The contract, in four cases:

Body Cleanup runs? Result
returns v yes return v
raises E yes re-raise E
cancelled yes re-raise CancelledError
bounded by cleanup_timeout never replaces the body’s outcome

When the body’s outcome is an exception and cleanup misbehaved, attach a note to that exception with add_note: exactly CLEANUP_TIMED_OUT if the bound expired, exactly CLEANUP_FAILED if cleanup raised. Both constants are in the starter. When the body succeeded, the value is the outcome and a cleanup problem is swallowed — there is no exception to annotate, and inventing one would destroy a result the caller earned.

The three things that make this hard

Cleanup must run even when the body was cancelled. asyncio’s cancellation is edge-triggered: cancel() delivers exactly one CancelledError, so an await in the handler suspends and resumes normally instead of re-raising immediately. That is what makes an awaiting cleanup possible at all — and it is also why cleanup can hang forever and silently ignore the cancellation you just accepted.

Cleanup must be bounded. async with asyncio.timeout(cleanup_timeout) around it. Note that asyncio.timeout cancels the current task on expiry and converts that CancelledError into TimeoutError — which can therefore only be caught outside the async with block. Catching it inside is the classic mistake.

CancelledError must always come back out. It is a BaseException precisely so except Exception cannot eat it. Inside the bounded cleanup, catch Exception — not BaseException — or you will swallow a cancellation that belongs to your caller. Every except BaseException in async code that does not end in raise is a cancellation bug.

What the report proves

The driver runs eight combinations of body outcome, cleanup behaviour and an external cancel.

  • log — the exact sequence of body:start, body:end, body:cancelled, cleanup:start, cleanup:end, cleanup:cancelled. Cleanup starts in every single case, including the cancelled one.
  • outcome / message / value — the body’s outcome survives. A cleanup that raises RuntimeError must never turn a ValueError into a RuntimeError, and must never turn a successful return into an exception.
  • notes — the note attached to the propagating exception, and nothing on the success path.
  • pending — no task left running. Note the hanging-cleanup cases: the timeout must actually cancel the cleanup coroutine, not merely stop waiting for it.

Where the type system earns its keep

with_cleanup is generic in the body’s result, so the caller keeps the real type through the wrapper. cleanup: Callable[[], Awaitable[None]] says the cleanup’s return value is not a result — anything it computes is discarded, and the None is the annotation that makes that explicit rather than implied.

Loading visualization…