Skip to content

← Concurrency Without Regret step 21 of 38

Medium Primitives

Deadline: one budget, many nested scopes

Give a call graph one budget instead of a dozen unrelated timeouts.

class Deadline:
    def __init__(self, budget: float) -> None: ...
    def remaining(self) -> float: ...
    def expired(self) -> bool: ...
    def scope(self, limit: float | None = None) -> asyncio.Timeout: ...

async def call_with_deadline[T](
    deadline: Deadline, op: Callable[[], Awaitable[T]], *, retries: int
) -> T
  • Deadline(budget) fixes an absolute deadline at budget seconds from now, on loop.time() — the same clock asyncio.timeout_at reads.
  • scope() returns a timeout scope expiring at that deadline. scope(limit) expires after limit seconds or at the deadline, whichever comes first. Nesting can only tighten, never extend.
  • call_with_deadline runs op inside the deadline, retrying RetryableError up to retries times. It must never start an attempt once the deadline has expired — raise TimeoutError instead. Any other exception propagates immediately.

Why the budget exists

Per-call timeouts do not compose. A one-second SLA, a two-second downstream timeout and five retries gives you a twelve-second request in which every individual call was “within timeout”. Nothing in that code is wrong line by line; what is missing is a single absolute point that every level shares.

What the report proves

  • attempts — the number of times op was entered. The budget case is the interesting one: an operation that consumes the whole budget on its first try must produce exactly one attempt, however many retries were configured. Retrying past an expired deadline shows up here immediately.
  • outcome / value / message — a fast success returns straight away; an exhausted retry count surfaces the last RetryableError; an expired budget raises TimeoutError.
  • expired — the deadline’s own view of itself afterwards.
  • level — for the two structural scenarios:
    • "outside" proves TimeoutError is raised outside the async with. asyncio.timeout cancels the current task and converts the CancelledError into TimeoutError in __aexit__, so a try/except TimeoutError inside the block never fires. This trips up nearly everyone once.
    • "inner" / "outer" proves nested scopes attribute the expiry to the level that actually blew, which is what Task.cancelling()/uncancel() bookkeeping buys you.

Where the type system earns its keep

scope() returns asyncio.Timeout — a real type, so async with deadline.scope(): type-checks and a scope() that forgot to return anything does not. call_with_deadline is generic in the operation’s result, so the retry wrapper is transparent to the caller’s types. And limit: float | None with None meaning “the whole remaining budget” is the honest signature: a sentinel 0.0 would be indistinguishable from a caller asking for no time at all.

Loading visualization…