concurrent.futures: ThreadPoolExecutor Done Properly

concurrent.futures is the right default for “run these blocking calls in parallel”. It is also the API with the three most expensive defaults in the standard library.

Failure one: the exception nobody observes

for item in items:
    pool.submit(handle, item)

Every one of those submit calls returns a Future. If handle raises, the exception is stored on the future and stays there. Nothing prints it. Nothing logs it. The loop above discards every future, so nothing ever calls .result() or .exception(), and the failure is simply gone — no traceback, no metric, no alert.

This is the same class of bug as a swallowed thread exception, but quieter: with a raw Thread, threading.excepthook at least prints to stderr. A discarded Future prints nothing at all.

The rule: every future you submit, you observe. Either keep them and call .result(), or use as_completed, or accept the failure explicitly with .exception().

Failure two: map()‘s two surprises

Executor.map looks like the ergonomic choice and behaves unlike map.

It consumes the input iterable immediately. All of it, before returning. Feed it a generator that reads a 10 GB file lazily and you have materialised every task at once — memory gone, and no backpressure whatsoever. Python 3.14 adds buffersize= to bound exactly this; before that, the only fix is to chunk the input yourself.

It raises at the wrong moment. The result is an iterator, and a worker’s exception is raised only when you reach that element. So

for result in pool.map(handle, items):
    write(result)

writes the first three results, then raises on the fourth — and the remaining tasks are cancelled at that point, so items 5..N never ran and you have no record of which ones. Half the work is done, half is not, and the exception tells you nothing about the boundary.

That is why “map over the items, tolerate failures” is written with submit and a dict from future to index. map cannot express partial success.

💡pool.map(handle, items) where handle raises on item 4 of 10, inside with ThreadPoolExecutor() as pool:. How many items ran? What do you know when the for loop raises? click to reveal

Somewhere between 4 and 10 ran, and you cannot tell which from the exception.

map submits everything up front, so all ten are queued and workers pick them up concurrently — items 5 through 10 may have completed, partially completed, or never started, depending on pool size and timing. When you reach element 4 the stored exception is raised; the iterator’s cleanup then cancels the futures that have not started, and lets the running ones finish.

So the state after the exception is: item 4 failed, items 0–3 succeeded and you consumed them, and items 5–9 are in an unknown mixture of done, running and cancelled. Nothing in the exception distinguishes them.

That is a genuinely bad place to be if handle has side effects — it is the difference between “retry the batch” and “retry the batch and double-charge six customers”. submit plus a future-to-index map gives you a per-item outcome and makes retry a decision rather than a gamble.

Failure three: sizing, containers, and nested submits

The default is min(32, (os.process_cpu_count() or 1) + 4). Python 3.13 changed the second term from os.cpu_count() to os.process_cpu_count(), which respects CPU affinity — a real improvement, and still not a cgroup quota. In a container limited to 0.5 CPU on a 64-core host, process_cpu_count() happily reports 64, and you get 32 threads fighting over half a core. For an I/O-bound pool, size from the downstream limit — the connection pool, the API’s rate limit — not from the CPU count.

And the deadlock the docs warn about explicitly: a task that submits to its own pool and waits for the result. With max_workers=4 and four tasks each blocking on a nested submit, all four workers are waiting for work that can only run on a worker. Nothing is running, nothing can start, and the pool sits there forever. It never happens in testing, because the pool is not saturated in testing.

💡shutdown(cancel_futures=True) — you call it during shutdown with 500 queued futures. What actually gets cancelled? click to reveal

Only the futures that have not started. Anything a worker has already picked up runs to completion; there is no mechanism to interrupt it, for the same reason there is no safe way to kill a thread.

So with max_workers=8, you cancel roughly 492 and wait for 8. That is usually what you want, and it is worth being precise about, because “cancel_futures” reads like “stop everything” and people size their shutdown grace period accordingly. Your real shutdown time is bounded by the slowest single task, not by the queue depth.

Note also that shutdown(wait=True) — the default, and what with does — still waits for the running ones even when you pass cancel_futures=True. If you want the queue cleared and the process gone, you need both flags and an acceptance that in-flight work is lost.

What good looks like

with ThreadPoolExecutor(max_workers=n) as pool:
    futures = {pool.submit(fn, item): i for i, item in enumerate(items)}
    for future, index in futures.items():
        error = future.exception()      # blocks; never re-raises
        ...

with guarantees shutdown(wait=True) on every exit path. submit plus an index gives per-item outcomes. And future.exception() — rather than a try: future.result() except — is the version that reads as “I am deciding what to do about failure”, which is the whole point.