Capstone: build a thread pool

Build a real thread pool out of std — workers, a shared channel, and a Drop that shuts it down cleanly — then run a batch of jobs through it and return the answers in input order.

pub fn pool_run(jobs: Vec<u64>, workers: usize) -> Vec<u64>

This is the concurrency capstone. It composes essentially every idea in the concurrency track: Send + 'static bounds on boxed closures, a shared receiver behind Arc<Mutex<..>>, channel disconnection as the shutdown signal, and Option<JoinHandle> as the classic move-out-of-Drop trick. Finishing it is the point at which you can honestly say you know std concurrency.

work and pool_run are given. Your job is Worker, ThreadPool, execute and Drop.

The pieces

type Job = Box<dyn FnOnce() + Send + 'static>;

Read that type slowly, because all four parts are load-bearing. FnOnce because a job runs once and may consume what it captured. Box<dyn ...> because every job is a different closure type and they have to live in one channel. Send because the value crosses a thread boundary. 'static because the pool has no idea how long its threads will live, so a job may not borrow anything with a shorter life than the program.

A ThreadPool owns a Vec<Worker> and an mpsc::Sender<Job>. All the workers share one mpsc::Receiver<Job> behind an Arc<Mutex<..>>, because mpsc is multi-producer, single-consumer — the receiver cannot be cloned, so the pool shares it instead. Each Worker owns a thread::JoinHandle<()> and loops: lock, receive, unlock, run.

Three things that decide whether this works

::: question Where exactly does the lock get released, and what happens if you get it wrong?

let message = {
    let guard = rx.lock().expect("poisoned");
    guard.recv()
};                  // <- guard dropped HERE, before the job runs
match message {
    Ok(job) => job(),
    Err(_) => break,
}

If you write while let Ok(job) = rx.lock().unwrap().recv() { job(); }, the temporary MutexGuard lives until the end of the whole while let expression — so every worker holds the lock while running its job, and no two jobs ever overlap. The pool compiles, passes every test, and is a slow single thread with extra steps.

This is the difference between “working” and “correct”, and it is invisible to a test suite that only checks answers. Clippy’s significant_drop_tightening (nursery, so not on by default) is the lint that looks for it; mostly, though, you have to know. :::

Shutdown order. recv() returns Err only when every sender has been dropped. So Drop for ThreadPool must drop the pool’s sender before joining the workers. Join first and the workers are still blocked in recv, waiting for a sender that you are holding — the program hangs forever, and no error message is ever printed. If your submission times out with no output, this is why.

Moving out of &mut self in Drop. Drop::drop gets &mut self, and JoinHandle::join takes self by value. The starter tries it directly and gets E0507: cannot move out of worker.handle which is behind a mutable reference. The standard fix is a small change to the struct that gives you something to leave behind — think about what Option lets you do that a bare field does not. It is one of the most-copied five lines in the Rust ecosystem, and it is worth deriving rather than looking up.

Determinism

Nothing here is timed and nothing is racy. work is the Collatz sequence length — a pure function — and pool_run writes each answer into slots[index], tagged by the job’s position in the input. The output is the same for one worker and for eight, which is the property the tests check: the same job list is run at three different widths and must produce byte-for- byte identical results.

If you find yourself wanting to return “the order they finished in”, stop: that is not reproducible, and a concurrent API that returns results in completion order is a concurrency API that cannot be tested. Tag by index.

One obsolete thing you may find online

Boxed FnOnce used to be uncallable — you could not move out of a Box<dyn FnOnce()>, so tutorials from before Rust 1.35 define a FnBox trait with fn call_box(self: Box<Self>). That is no longer needed: Box<dyn FnOnce()> is directly callable today. If a tutorial hands you self: Box<Self>, it is describing a language that no longer exists.