Skip to content

← Error Handling with Exceptions, Retries and Warnings step 14 of 20

Medium End-to-End

Retry a Fetch with Exponential Backoff

Build a retry helper like the one in Retry Failed Calls with Backoff, and a solve function that reports what it did. solve fakes the network with a script of results and records each wait instead of sleeping, so the tests finish at once.

solve(url="https://pay/charge", script=["transient", "ok"], attempts=3)
-> {"outcome": "ok", "body": b"payload for https://pay/charge", "calls": 2,
    "sleeps": [0.5], "type": None, "message": None, "cause_type": None}

Here TransientError is worth retrying, and TerminalError is not.

Write fetch_with_retry

Fill in fetch_with_retry(fetch, url, attempts, sleep), using the starter's BASE_DELAY = 0.5. The limit counts every attempt, including the first.

  1. If attempts < 1, raise ValueError(f"attempts must be at least 1, got {attempts}") before you call fetch or sleep.
  2. Call fetch(url). If it returns, return its bytes unchanged.
  3. If it raises TransientError and attempts remain, call sleep(BASE_DELAY * 2 ** (n - 1)), where n is the number of the attempt that failed, counting from 1. Then try again.
  4. If the last allowed attempt raises TransientError, raise RetryExhaustedError from that same exception, with the message giving up on {url} after {attempts} attempts. Write attempt, singular, when attempts is 1.
  5. Let any other exception from fetch escape at once. An exception from sleep also escapes, with no further fetch, even if it is a TransientError.

Never sleep after a success or after the last failure. Four failures with four attempts request the waits [0.5, 1.0, 2.0], 3.5 seconds in all.

Write solve

solve(url, script, attempts) builds a fake fetch and a recording sleep, then calls fetch_with_retry. When attempts is positive, script has at least one entry. Each call to the fake fetch uses the next entry, and once the script runs out it repeats the last one. With n as the call number, counting from 1:

  • "transient" raises TransientError(f"connection reset on call {n}").
  • "terminal" raises TerminalError(f"404 on call {n}").
  • Any other entry returns f"payload for {url}".encode().

The fake fetch counts every call, including the ones that fail. The fake sleep appends its delay to a list. When attempts is zero or less, the script may be empty, so read it only inside the fake fetch.

Catch Exception around the call to fetch_with_retry to build the failure report. Inside fetch_with_retry, catch only TransientError from fetch.

What solve returns

Key Value
"outcome" "ok" on success, otherwise "raised"
"body" The returned bytes on success, otherwise None
"calls" The number of fetch calls
"sleeps" The requested waits, in order
"type" The escaping exception's class name, otherwise None
"message" str(exc) on failure, otherwise None
"cause_type" The class name of the exception's __cause__, or None

The tests show the expected body in a base64 form. Return plain bytes, and the runner converts them.