What async fn actually compiles to

Calling an async fn does not run its body. The call builds a value, and the body runs later, one piece at a time, whenever something polls that value. This article describes the shape of that value. Four common surprises in async Rust follow from it: these futures are not Unpin, a recursive async fn needs a heap allocation, a lock guard held across an .await keeps the lock locked, and futures that await other futures can become large.

Three terms first. A future is a value whose type implements std::future::Future. Polling a future means calling its poll method, which returns Poll::Ready(value) when the work is finished or Poll::Pending when it has to wait for something. A suspension point is a place where the body can stop and return Pending to its caller; in an async fn, every .await is one.

The signature

The Rust Reference describes an async fn as an ordinary function whose body is wrapped in an async move block. For example,

async fn example(x: &str) -> usize {
    x.len()
}

is roughly equivalent to

fn example<'a>(x: &'a str) -> impl Future<Output = usize> + 'a {
    async move { x.len() }
}

Read the second version literally. example is a normal function. Calling it evaluates the arguments and moves them into the returned value, but runs no statement of the body. The returned type is generated by the compiler and has no name you can write; code outside can only refer to it as "some type that implements Future<Output = usize>". The + 'a says that the future keeps the borrowed argument inside it, so the future cannot outlive the string it was given.

The body runs only when the future is polled. When the body finishes, poll returns Poll::Ready with the function's return value. Polling the future again after that is a bug, and the compiler-generated future panics if it happens.

This kind of future is a stackless coroutine. A coroutine is code that can stop partway through and later continue where it stopped. Some systems, such as the goroutines in Go, give each coroutine its own call stack and keep that stack while the coroutine is suspended; those are called stackful. A Rust future has no stack of its own. When it suspends, the values it will need later are saved inside the future value, and poll returns normally to its caller. The future's size is a fixed number of bytes known at compile time.

A model: one state per suspension point

Take this function, where step is some other async fn taking and returning a u32:

async fn pipeline(n: u32) -> u32 {
    let a = step(n).await;
    let b = step(a + 1).await;
    a + b
}

The future can be in one of four places: not started yet, waiting at the first .await, waiting at the second .await, or finished. At each place it must remember the values that the rest of the body will still use. A hand-written version could store those places as an enum, where StepFuture stands for the type of the future step returns:

enum PipelineState {
    Start { n: u32 },
    AwaitingA { fut: StepFuture },
    AwaitingB { a: u32, fut: StepFuture }, // `a` is kept: the last line needs it
    Done,
}

AwaitingA does not keep n, because nothing after the first .await reads it. AwaitingB keeps a, because the last line adds it to b.

A poll method for this enum would loop over a match on the current state. It runs code until it reaches the next .await and polls the inner future there. If the inner future returns Pending, poll saves the current state and returns Pending. If the inner future returns Ready, poll moves to the next state and keeps going, so one call to poll can pass through several states.

This enum is a model, not the compiler's output. rustc does not generate a Rust enum with these names. It lowers async bodies to coroutines in its MIR intermediate representation and chooses their memory layout itself. Two details of the real layout matter in practice:

• Values that are never stored at the same time can share the same bytes, so the size is roughly the largest state rather than the sum of all states. It is not always the smallest size possible.
• The compiler decides what to save by analysing the code, and it can keep a value longer than the model suggests. A value whose type has a destructor, such as a lock guard, stays in the future until the end of its scope, because its destructor runs there, unless it is moved out earlier (for example by drop). A local that has been borrowed is also kept while it is in scope.

The model is still enough to predict everything below. The exercise hand-compile an async fn into a state machine has you write this kind of enum yourself and check that it behaves like the async fn.

What .await expands to

The Reference describes expr.await with code like the following. It is not valid Rust you can compile: cx stands for the Context that the enclosing future was polled with, and yield is the coroutine operation that stable Rust does not let you write directly.

match IntoFuture::into_future(expr) {
    mut fut => loop {
        let pinned = unsafe { Pin::new_unchecked(&mut fut) };
        match pinned.poll(cx) {
            Poll::Ready(value) => break value,
            Poll::Pending => yield Poll::Pending,
        }
    }
}

Three details are worth noticing:

• It calls IntoFuture::into_future first. Every future implements IntoFuture, and other types can implement it too, so you can .await a value that only becomes a future when awaited.
• yield returns Pending from the enclosing future's poll. The next poll of the enclosing future resumes inside this loop and polls fut again. If fut is still not ready, for example because the enclosing future was polled for some unrelated reason, it returns Pending again and the loop yields again.
• Pin::new_unchecked is unsafe because it promises that fut will not move again. That promise holds because fut is stored inside the enclosing future, which is itself pinned while it is being polled. The next sections explain why pinning is needed.

Four consequences

1. The future itself is not a heap allocation

The future is one value of fixed size, and an inner future it awaits is stored inside it as a field. Calling pipeline(3) produces that value wherever you put it: in a local variable, in a Vec, or inside another future. Creating it does not allocate, and polling it allocates only if code inside the bodies being run allocates.

Allocation still happens in two other ways. The body can allocate, for example by creating a Vec. And code that stores futures can box them, for example with Box::pin. Many executors put each spawned task in its own heap allocation; whether a particular spawn or join function allocates depends on the library that provides it.

2. Async futures are not Unpin

In this example, yield_once stands for any future that returns Pending once and Ready the next time it is polled:

async fn first_byte_later() -> u8 {
    let data = [1u8, 2, 3, 4];
    let first = &data[0]; // points at `data`, which is stored in the future
    yield_once().await;   // `data` and `first` are both saved here
    *first
}

While the future waits at the .await, it stores the array data and the reference first, which points at data. So the future contains a pointer to one of its own fields. A type like this is called self-referential.

If the future were moved to a new address after that point, its bytes would be copied but first would still hold the old address. The next poll would read memory the future no longer occupies. (If data were a Vec<u8> instead, &data[0] would point into the vector's heap buffer, which does not move when the future moves. The array is what makes this example self-referential.)

Rust prevents this with Pin. Future::poll takes self: Pin<&mut Self> rather than &mut self. For a type that does not implement the Unpin trait, holding a Pin<&mut T> comes with a guarantee that the value will stay at the same address until it is dropped. Safe code cannot move a value out of a Pin<&mut T> unless T: Unpin. The compiler does not check whether a particular async body actually holds a reference across an .await; it treats every async fn and async block future as not Unpin. The article why Pin and Unpin exist covers this in detail.

The guarantee starts when the future is pinned. Before its first poll, a future has run none of its body and holds no references into itself, so moving it is fine. That is why you can pass first_byte_later() by value to a block_on function: block_on receives the future, pins it, and only then polls it.

3. A recursive async fn needs indirection

Suppose async fn f awaits a call to f. While it waits at that .await, the future of the outer call stores the future of the inner call. Both have the same type, so the type would have to contain a field of its own type, and no finite size works.

rustc 1.98.1 reports this as:

error[E0733]: recursion in an async fn requires boxing
  = note: a recursive `async fn` call must introduce indirection such as `Box::pin` to avoid an infinitely sized future

The fix is to store the inner future behind a pointer. Box::pin(f(n - 1)).await is accepted, because a Box is the same size no matter how large the value it points to is. Each level of recursion that is in progress then costs one heap allocation, so deep recursion means many allocations. The exercise async recursion, closures and streams practises this.

4. Values held across an .await are stored in the future

let guard = mutex.lock().unwrap();
do_something(&guard);
other_service().await; // guard is still alive here
use_it(&guard);

guard is still in scope at the .await, so it is saved inside the future, and the lock stays locked while the future waits. The lock is released only when the guard is dropped: after the body resumes and the guard goes out of scope, or when the whole future is dropped. If the future is never polled again and never dropped, the lock is never released. This is true even if use_it(&guard) were removed, because the guard's destructor runs at the end of its scope.

That causes three kinds of problem. Other code that needs the lock has to wait, and with std::sync::Mutex that wait blocks a whole thread. If the awaited work needs the same lock, the program can deadlock. And std::sync::MutexGuard does not implement Send, so the future does not implement Send either. Functions that require a Send future, such as the spawn functions of multi-threaded executors, reject it. The compiler's error names the guard: it says the future is not Send, that MutexGuard is the value that is not Send, and points at the .await where it is held.

The usual fix is to put the locked section in its own block that ends before the .await:

let value = {
    let guard = mutex.lock().unwrap();
    do_something(&guard);
    *guard
}; // guard dropped here, so the lock is released
other_service().await;

Calling drop(guard) before the .await also releases the lock at that point. However, if the guard was borrowed earlier (do_something(&guard) or *guard both borrow it), rustc 1.98.1 still counts it as held across the .await when deciding whether the future is Send, because it keeps borrowed locals in that analysis until the end of their scope. The block form avoids both problems.

Clippy has two lints for this pattern: clippy::await_holding_lock for lock guards and clippy::await_holding_refcell_ref for RefCell borrows. Both are in the suspicious group, which Clippy enables by default. The exercise blocking and locks across await practises avoiding this pattern.

Check your understanding Each async fn's future is roughly as large as its largest state. Suppose outer awaits middle, which awaits inner, and each one keeps a 1 KiB array alive across its .await. About how large is outer's future? Show answer Hide answer

About 3 KiB. While outer waits, it stores its own array plus middle's whole future, and middle's future stores its own array plus inner's future. The sizes add up along the chain. Measured with rustc 1.98.1 on a 64-bit target, with inner awaiting a one-byte future, inner is 1026 bytes, middle 2051 and outer 3076: each array plus a few bytes of state information.

Large futures cost time when they move, because moving a value copies all of its bytes. Passing a future by value to spawn, pushing it into a Vec, and returning it from a function are all moves. They can also use a lot of stack space when they are created or moved on the stack.

Two fixes follow from the model. First, keep less alive across each .await: if the array is only needed before the .await, use it inside a block that ends before the .await, and the future no longer stores it. Second, await an inner future through Box::pin. The outer future then stores only a pointer, at the cost of one heap allocation.

Clippy's clippy::large_futures lint warns about futures above a size threshold (16384 bytes by default). It is in the pedantic group, so it is not enabled by default; you can turn it on explicitly.

Where this leads

The Pin and Unpin article explains the guarantee behind consequence 2, and the hand-compiled state machine exercise has you write the model enum and show that it matches the async fn.