Why async exists: blocking, threads, and the cost of waiting

Before you write a line of async Rust, it is worth being precise about what problem it solves — because the two most common mistakes with async are both mistakes of expectation, not of syntax.

The first is reaching for async on CPU-bound work. The second is reading .await as “run this in parallel”. Both come from the same place: an unexamined belief that async is a performance feature you turn on.

It is not. Async is a concurrency model with real costs, and its one genuine superpower is multiplexing a very large number of mostly-idle waits onto a very small number of threads.

What a blocked thread actually costs

Start with the thing async replaces.

When a thread calls read() on a socket with no data, the kernel takes it off the run queue and it sits there. It is not consuming CPU. But it is not free either:

• Address space. Each thread reserves a stack — commonly 8 MiB of virtual address space on Linux, 1 MiB on Windows, with only the touched pages actually committed. Ten thousand threads is tens of gigabytes of reserved address space and a real amount of resident memory for the parts that get touched.
• A kernel object. A task struct, a scheduling entity, an entry in every list the scheduler walks.
• Context switches. Every wakeup is a trip through the kernel and a cache flush that your working set does not enjoy.

None of that matters at ten threads. At ten thousand it is the whole problem. The historical name for the resulting wall is the C10K problem — how do you serve ten thousand concurrent connections on one box — and every async runtime that exists is an answer to it.

💡A web server holds 10,000 open connections. On average each one is idle 99.9% of the time. How many threads does it need, and how many does it need if it uses one thread per connection? click to reveal

It needs roughly ten threads’ worth of actual work — 10,000 connections at 0.1% utilisation is ten connections’ worth of CPU at any instant.

One thread per connection gives it ten thousand threads to do that: ten thousand stacks, ten thousand scheduler entities, and a context switch every time any one of them gets a byte. The work is trivial; the bookkeeping is enormous.

That gap — between the work you have and the machinery you need to hold it — is exactly what async collapses. Ten thousand futures parked in a HashMap cost ten thousand small structs and no kernel objects at all.

Concurrency is not parallelism

These two words get used interchangeably and they mean different things.

Concurrency is dealing with many things at once: your program has several logical activities in flight, and it interleaves progress on them. One cook making three dishes, moving between them while things simmer.

Parallelism is doing many things at once: two or more physical execution units running simultaneously. Three cooks.

You can have either without the other. A single-threaded event loop is concurrent and not parallel. A rayon parallel map over an array is parallel and, arguably, not usefully concurrent. Async gives you concurrency. If you want parallelism you need threads underneath — which is what a multi-threaded runtime provides, and which is a separate decision from using async at all.

The single-threaded executor you build in item 16.17 makes this concrete: three tasks make interleaved progress on one thread, and the output proves it.

CPU-bound versus I/O-bound

The other axis, and the one that decides which tool you want.

I/O-bound work spends most of its time waiting: for a socket, a disk, a database, another service. The CPU is idle; the thread is parked. This is where async wins, because parking ten thousand futures is cheap and parking ten thousand threads is not.

CPU-bound work spends its time computing. There is no wait to multiplex. Async gives you nothing here — worse than nothing, because a CPU-bound task on an async runtime holds its worker thread hostage and blocks every other task on that thread. (Item 16.22 makes you watch it happen.) Threads are the right answer for CPU-bound work, and a thread pool sized to your core count is both faster and dramatically simpler.

The honest summary:

	few concurrent things	tens of thousands
CPU-bound	just do it	threads / rayon
I/O-bound	threads are fine, and simpler	async

Notice how much of that table is not async. For most programs, std::thread is the better engineering choice, and reaching for async because it sounds modern is how you end up with a slower program and a harder codebase.

Never say “async is faster”

It is worth being disciplined about this phrase, because it is wrong in a way that misleads for years.

For a single request, async is usually slightly slower than a blocking call on a thread: there is a state machine to poll, a waker to invoke, a scheduler to consult. What async buys is scale in the number of concurrent waits per thread. Throughput at high concurrency, not latency at low concurrency.

Benchmarks that show async winning are almost always measuring the regime where the thread-per-connection version has fallen over. Benchmarks that show threads winning are almost always measuring the regime where there is nothing to multiplex. Both are correct.

What Rust actually ships

Here is the fact that shapes this entire track.

The Rust standard library contains:

• the async and .await syntax, handled by the compiler;
• the Future trait, in std::future;
• Poll, Context, Waker, Wake, RawWaker in std::task;
• pin!, Pin, Unpin in std::pin.

The Rust standard library does not contain:

• an executor;
• a reactor;
• a timer;
• an async TCP socket;
• a #[main] attribute that starts any of the above.

Calling an async fn in a fresh cargo new project and doing nothing else produces a future that is never polled and a #[must_use] warning. That is the entire runtime story in std.

💡Why would a language ship async syntax and then refuse to ship anything that can run it? Doesn't that just guarantee everyone depends on the same third-party crate anyway? click to reveal

Because a scheduler is a policy, and there is no policy that suits every target.

A microcontroller with 64 KiB of RAM and no allocator wants a fixed set of tasks in static memory with an interrupt-driven reactor (embassy). A kernel wants something that never allocates on the wakeup path. A web server wants work-stealing across all your cores with an epoll loop. A GUI wants everything on the main thread, because the toolkit demands it. A test wants a deterministic single-threaded executor with a fake clock — which is exactly what you build in item 16.19.

Baking any one of those into std would make Rust unsuitable for the others, and std is meant to work everywhere from bare metal upward. Rust already tried imposing a runtime on every program: green threads and a segmented-stack scheduler were in the language before 1.0, and they were removed precisely because they taxed programs that did not want them.

So the split is deliberate: the language standardises the interface — one Future trait, one Waker contract — so that libraries written against it compose, and leaves the policy to crates. The cost is real (you do pick a runtime, and mixing two in one binary is a known source of pain), and the alternative was judged worse.

There is also a happier consequence, and it is why this track can exist at all: since nothing is hidden inside the language, you can build the missing half yourself, out of std, in about sixty lines.

What you are about to do

Because there is no runtime here — no tokio, no smol, no crates at all — you cannot treat the executor as magic. Every async problem in this track is driven by a driver that lives in your own file.

You will write, in order: a future by hand, a block_on, a waker, an executor, a join, a select, a JoinHandle, and a reactor with timers. By the end you will have a working two-part async runtime built entirely from std, and tokio’s source will look like a thing you recognise rather than a thing you use.

That is a better way to learn async Rust than #[tokio::main], and it is the only way available here.