Your script downloads 20 web pages, one after another. Each download takes half a second, so the script takes 10 seconds. For almost all of that time, your program does nothing. It has sent a request and is waiting for the server to answer.
Concurrency means making progress on several tasks over the same period. While one download waits for its server, another one can start. Twenty downloads that each wait half a second can finish in about half a second in total.
Find out whether you are waiting or computing
Before you pick a tool, find out why your code is slow. There are two kinds of slow code.
- I/O-bound code is slow because it waits for something outside the program, such as the network, a disk or a database. Downloads, API calls and database queries are I/O-bound.
- CPU-bound code is slow because the processor is busy computing. Parsing big files, resizing images and number crunching are CPU-bound.
To tell them apart, measure the slow part two ways, by the wall clock and by the CPU time your program used.
import time
wall = time.perf_counter()
cpu = time.process_time()
run_the_slow_part()
print(f"wall: {time.perf_counter() - wall:.2f}s")
print(f"cpu: {time.process_time() - cpu:.2f}s")
If the CPU time is much smaller than the wall time, the code is mostly waiting. If the two are close, it is mostly computing.
Three tools, three jobs
Python gives you three ways to do several things at once.
| Tool | Good for | How it works | Main catch |
|---|---|---|---|
Threads (threading, ThreadPoolExecutor) |
Waiting, with ordinary libraries | Several threads inside one program, sharing its memory | Shared data needs a lock |
Processes (ProcessPoolExecutor, multiprocessing) |
Computing, on several CPU cores | Several separate programs, each with its own memory | Data is copied between them |
| asyncio | A lot of waiting, with async libraries | Many tasks in one thread, taking turns at each await |
Every slow call must be async |
Threads cannot speed up Python code that computes. A lock inside the interpreter, the GIL, lets only one thread run Python at a time. A later article in this track, "Threads, Processes and the GIL", shows why.
Run your first thread pool
A thread pool is a fixed group of worker threads that take jobs from a
queue. concurrent.futures.ThreadPoolExecutor gives you one. Here are the 20
downloads, five at a time:
from concurrent.futures import ThreadPoolExecutor
from urllib.request import urlopen
def download(url: str) -> bytes:
with urlopen(url, timeout=10) as response:
return response.read()
urls = [f"https://example.com/page/{n}" for n in range(20)]
with ThreadPoolExecutor(max_workers=5) as pool:
pages = list(pool.map(download, urls))
pool.map(download, urls) works like the built-in map. It calls
download once per URL and gives back the results in the same order as
urls. The difference is that up to five calls run at the same time. When
the with block ends, the pool waits for every job to finish and then
stops its threads.
max_workers sets how many calls run at once. More workers give more
overlap, but also more open connections to the server. Pick a number that
the server and your machine can handle.
Check your understanding Your script spends 9 of its 10 seconds resizing images with pure Python code. Will a thread pool make it faster? Show answer Hide answer
No, not on a standard Python install. Resizing images in pure Python is CPU-bound, and threads take turns running Python code. Use a process pool to spread the work over several cores, or a library that does the resizing in C, such as Pillow.
What this track covers
The track starts with threads, because they work with the libraries you already use:
- Thread pools, futures, and collecting results and errors.
- The GIL, and when to use processes instead.
- Race conditions and locks, for when threads share data.
- Coordinating threads and stopping them cleanly.
Then it moves to asyncio. You learn coroutines and the event loop, how to run
many tasks with a TaskGroup, cancellation and timeouts, queues, and how to
shut down a service. It ends with a capstone, an async worker pool with
retries and clean shutdown.