Choose the Right Concurrency Tool for Each Job

You have now used most of Python's concurrency tools one at a time. A real service often needs several at once. Take an order service. For each order, it fetches the customer and the cart from two HTTP APIs. It also asks a payment SDK for an invoice and computes a fraud score in pure Python. Each part needs a different tool:

import asyncio
import time
from concurrent.futures import ProcessPoolExecutor


async def fetch(what: str) -> str:
    await asyncio.sleep(0.5)  # stands in for an async HTTP call
    return what


def create_invoice(order_id: int) -> str:
    time.sleep(0.5)  # stands in for a blocking SDK call
    return f"invoice-{order_id}"


def fraud_score(cart: str) -> int:
    return sum(range(5_000_000)) % 100  # stands in for heavy pure-Python work


async def handle(order_id: int, cpu_pool: ProcessPoolExecutor) -> tuple[str, str, int]:
    async with asyncio.TaskGroup() as tg:
        customer = tg.create_task(fetch("customer"))
        cart = tg.create_task(fetch("cart"))
    invoice = await asyncio.to_thread(create_invoice, order_id)
    loop = asyncio.get_running_loop()
    score = await loop.run_in_executor(cpu_pool, fraud_score, cart.result())
    return customer.result(), invoice, score


async def main() -> None:
    with ProcessPoolExecutor() as cpu_pool:
        print(await handle(42, cpu_pool))


if __name__ == "__main__":
    asyncio.run(main())

The two fetches wait at the same time in a TaskGroup. The blocking SDK call waits in a worker thread, so the event loop keeps serving other orders. The fraud score runs in another process, which has its own GIL and can use another core.

Match each part to a tool

Ask what each part of the job does, then pick the tool from this table.

The part of the job	The tool	Why it fits
Waits on an async library	Tasks in a `TaskGroup`	The waits overlap, and one failure cancels the rest
Waits on a library that only blocks	`asyncio.to_thread`, or a `ThreadPoolExecutor` in code with no event loop	The wait happens in a thread, and the GIL is free while it waits
Computes in pure Python	A `ProcessPoolExecutor`	Each process has its own GIL, so the work spreads over several cores
Changes data that threads share	One `threading.Lock` around the check and the update	No other thread can get between the check and the update
Produces work faster than it is consumed	A bounded `asyncio.Queue` and a fixed set of workers	`put` waits when the queue is full, so the producer slows down
Must finish within a time limit	`asyncio.timeout`	It cancels the wait when time runs out

Each tool links back to its article in this track. Pick the simplest mix that covers your parts, then measure it on realistic work. If every part waits on async libraries, asyncio alone is enough.

Connect the models at a few clear points

When a design mixes tasks, threads and processes, pass work between these models only at a few clear points.

Keep blocking calls off the event loop's thread. Use asyncio.to_thread, or loop.run_in_executor with a pool you choose, as the example does.
From another thread, reach the event loop only through loop.call_soon_threadsafe or asyncio.run_coroutine_threadsafe. Objects such as asyncio.Queue are not thread-safe.
Give every task, thread and pool an owner that waits for it and sees its errors, such as a TaskGroup, a with block around an executor, or a join on a thread.

Plan how each part stops

Each model stops in its own way, and the way it stops often decides the design.

A task stops at its next await after you cancel it. Its finally blocks run, and they can take time. Code that never awaits cannot be cancelled.
A thread keeps running until it returns, and nothing outside it can kill it. Give it a stop signal it checks, such as a threading.Event, and give its blocking calls their own timeouts.
A process ends when you call terminate() or kill() on it, but it then skips its cleanup and may leave work half done. Use a process for work that must stop on time and is safe to retry.

Check your understanding A handler runs a 2-second pure-Python report with await asyncio.to_thread(build_report, data). Four requests arrive at once. Why do all four take about 8 seconds to finish? Show answer

to_thread keeps the event loop running, but the four threads take turns on the GIL. Together they take as long as running the reports one after another. Run build_report in a ProcessPoolExecutor through loop.run_in_executor instead. With four free cores, the four reports then run at the same time.

Check your understanding You wrap await asyncio.to_thread(download, url) in async with asyncio.timeout(5):. The timeout fires. Has the download stopped? Show answer

No. The timeout cancels your await, and your code moves on. The thread keeps running download until it returns. To stop the download on time, give it its own timeout, such as the HTTP client's timeout argument. Or pass it a threading.Event that it checks between chunks.

Going further

The next track, on parallelism, goes deeper into three more options.

InterpreterPoolExecutor, new in Python 3.14, runs each worker in its own interpreter inside one process.
The free-threaded build of Python 3.13 and 3.14 drops the GIL, so threads can run Python code on several cores at once. Shared data still needs locks there.
NumPy and similar libraries release the GIL inside many operations, so threads can speed them up. Some operations already use several cores, and adding your own workers on top can slow them down.