Why macros exist: code that writes code

Rust code uses many macros: println!, vec!, assert_eq!, format!, matches!, write! and todo! are all macros. The ! after the name marks a macro invocation. This article explains what a macro is, what it can do that a function call cannot, how Rust's macros differ from C's, and when a function or trait is the better tool.

What a macro is

A macro is a rule the compiler applies while it builds your program. The compiler takes the tokens written inside the invocation (for vec![1, 2, 3], the tokens 1, ,, 2, , and 3) and the macro turns them into new Rust code that replaces the invocation. This replacement is called expansion. Type checking happens later, on the expanded code.

A function works differently. Its body is compiled once, and each call passes values to it while the program runs.

Four things a function call cannot do

1. Accept different numbers of arguments of different types.

fn main() {
    let name = "Ada";
    let points = 42;
    let game = "chess";
    println!("{} scored {} in {}", name, points, game); // Ada scored 42 in chess
    println!("done");
}

The first println! receives a format string and three values of two different types; the second receives only a format string. An ordinary Rust function has a fixed list of parameters, each with a declared type. Generics let one parameter accept many types, and a tuple or slice can bundle several values into one argument, but no Rust function signature accepts three loose arguments at one call and none at another.

There is one narrow exception. A declaration of a C function, such as printf, inside an extern block may end in ... to accept extra arguments. Calling it is unsafe, and the compiler does not check that the extra arguments match what the C function expects. Item 18.27 covers this. Defining a variadic function in Rust itself is not available on stable Rust.

println! also checks its format string while compiling. It passes the string to the built-in format_args! macro, which compares the {} placeholders with the arguments: too few arguments, or an argument that no placeholder uses, is a compile error. Each argument printed with {} must implement the Display trait; an argument whose type does not, such as a Vec<i32>, is rejected with error E0277.

2. Accept syntax that is not an expression.

fn main() {
    let buffer = vec![0u8; 1024];
    let x = Some(5);
    let big = matches!(x, Some(n) if n > 3);
    println!("{} {}", buffer.len(), big); // 1024 true
}

Every argument to a function must be an expression, something that produces a value. The input 0u8; 1024 is not an expression. It is syntax that vec! defines for itself, meaning "a vector of 1024 copies of 0u8". In matches!, the second input Some(n) if n > 3 is a pattern followed by a guard (an extra if condition), the same syntax as one arm of a match. A pattern is not a value, so it cannot be passed to a function. matches! expands into a match expression that uses the pattern, which here produces true because 5 > 3.

One macro can also accept several unrelated input shapes. vec![], vec![1, 2, 3] and vec![0u8; 1024] all invoke the same macro; each shape is handled by a different rule inside it.

3. Generate items.

An item is a definition that can appear at the top level of a module: a function, struct, enum, trait, impl block, static, module or macro definition. A function call is an expression. It produces a value and cannot add definitions to the program. A macro can expand into items.

#[derive(Debug)] on a struct invokes a derive macro that writes an impl Debug for YourType block from the struct's definition, so nobody has to type it. thread_local! expands into an item of type std::thread::LocalKey (with Rust 1.98.1 on a typical desktop target, a const item) plus internal definitions that create and store each thread's copy of the value. The exact expansion is a standard library detail that can differ between platforms and Rust versions.

4. Change control flow at the call site, or read the call's source text.

A macro's expansion is placed inside the calling function. A return, break or ? in the expansion therefore acts on the caller's function or loop. A return inside a function only leaves that function. The stringify! macro turns its input tokens into a string literal without evaluating them; assert! uses the same idea to put the failing condition's text in its panic message.

macro_rules! ensure_positive {
    ($value:expr) => {
        if $value <= 0 {
            return Err(format!("{} must be positive", stringify!($value)));
        }
    };
}

fn area(width: i32, height: i32) -> Result<i32, String> {
    ensure_positive!(width);
    ensure_positive!(height);
    Ok(width * height)
}

fn main() {
    println!("{:?}", area(3, 4)); // Ok(12)
    println!("{:?}", area(3, -1)); // Err("height must be positive")
}

The second call to area stops at ensure_positive!(height): the expanded return leaves area itself, and stringify!($value) produces the text height that was written at the call site.

These four abilities are the usual reasons to write a macro. When a function, generic function or trait can do the same job, it is usually the clearer choice; the last section explains why.

How macro_rules! differs from the C preprocessor

In C, a macro defined with #define is expanded by the preprocessor, a translation step that runs before the C compiler parses expressions. The preprocessor replaces each use of the macro with its replacement tokens and substitutes each argument as a plain sequence of tokens. It does not know which tokens form an expression, so operator precedence can change the meaning:

#define DOUBLE(x) x * 2
DOUBLE(1 + 2)   /* becomes 1 + 2 * 2, which is 5, not 6 */

C programmers avoid this by wrapping each parameter, and the whole replacement, in parentheses.

Rust's macro_rules! macros also work on tokens, but with two differences that matter here.

First, the input is organized into token trees. A token tree is either a single token (an identifier, literal, operator or punctuation mark) or a group of token trees enclosed in a matching pair of (), [] or {}. Unbalanced delimiters are rejected before any macro runs, and a macro's output is also made of token trees, so expansion cannot produce an unmatched bracket.

Second, a macro can require a captured piece of input to be parsed as a particular kind of syntax, using a fragment specifier. $x:expr requires an expression, and the captured expression stays grouped as one unit when it is substituted:

macro_rules! double {
    ($x:expr) => {
        $x * 2
    };
}

fn main() {
    println!("{}", double!(1 + 2)); // 6
}

Here double!(1 + 2) computes (1 + 2) * 2. A macro that captures raw token trees instead, with the matcher $($t:tt)*, and writes them before * 2 produces the C result, 5. Item 18.4 asks you to repair exactly that.

Macro expansion happens before type checking. A macro_rules! macro sees only the tokens it is given. It cannot ask what type an expression has, whether a type implements a trait, or what fields a struct has when it receives only the struct's name. Procedural macros (items 18.16 to 18.19) also run before type checking and also receive tokens, so they have the same limit on type information. What they add is that the macro is ordinary Rust code that can examine and build token sequences in any way, which is how a derive macro reads the fields of the struct it is attached to.

Comparison with Python decorators and Java annotation processors

A Python decorator runs while the program runs, when the decorated def statement executes; it receives a function object and returns a replacement object. A Rust macro runs during compilation and receives tokens. Expansion itself costs nothing when the program runs, but the code a macro produces runs like any other code.

A Java annotation processor runs during compilation, like a Rust macro, but the standard processing API lets it generate new source files rather than rewrite the annotated code. A Rust attribute macro, by contrast, replaces the item it is attached to with its own output.

Vocabulary

• Invocation: a place where a macro is used, such as vec![1, 2, 3]. Function-like macros are invoked with !, which tells a reader that the contents may follow rules other than a function call's. Attribute macros (#[name]) and derive macros (#[derive(Name)]) are invoked without !.
• Rule: one matcher => transcriber pair inside macro_rules!. A macro may have several rules; the compiler tries them in order and uses the first one whose matcher fits the input.
• Matcher: the left side of a rule, a pattern that the invocation's token trees are compared against.
• Transcriber: the right side of a rule, the tokens produced when the rule matches, with captured pieces substituted in.
• Expansion: replacing an invocation with the transcriber's output, and also the name for the resulting code.
• Fragment specifier: the :expr, :ident, :ty and similar suffixes that say what kind of syntax a captured piece must be (item 18.3).
• Hygiene: the rules that keep local variable names and labels introduced inside a macro separate from the same names at the call site (item 18.11).

An invocation is not a call at run time. println!("hi") expands into code that calls a printing function in the standard library, so a debugger or a backtrace shows that function and the code around the invocation, not a function named println.

When to use a macro

A macro has costs that a function does not. Error messages can refer to the macro's rules or its expanded code instead of pointing directly at the caller's mistake. Editor features such as completion, go-to-definition and inline type hints may work less well inside macro input, because the editor has to expand the macro to understand it. Readers also have to learn each macro's own input syntax.

A common order of preference is therefore:

1. a plain function;
2. generics and traits, when the same logic must work for several types;
3. macro_rules!, when you need one of the four abilities above;
4. a procedural macro, when the generated code depends on the structure of an item, such as the fields of a struct.

Repeated code is not by itself a reason to write a macro. A helper function, a generic function or a trait with a default method often removes the repetition while keeping ordinary type checking. Item 18.20 returns to this choice with worked examples.

Where this track goes

Items 18.2 to 18.15 build macro_rules! skills from a first rule up to an arithmetic macro with operator precedence, covering repetition, hygiene, follow-set rules (which tokens may come after a captured fragment) and the recursion limit.

Items 18.16 to 18.20 are articles about what macro_rules! cannot do and what procedural macros do about it. They are articles rather than exercises because a procedural macro must be compiled as its own crate with the proc-macro crate type, so it cannot be defined in the crate that uses it, and each exercise here is a single source file.

Items 18.21 to 18.27 cover calls between Rust and C. On typical Linux and macOS targets the Rust standard library already links the platform's C library, so a single-file program can declare C functions such as strlen, qsort and snprintf in an unsafe extern "C" block and call them without extra crates, build scripts or linker flags. Other targets can differ: on Windows with the MSVC toolchain, for example, snprintf needs an additional library, and embedded targets may have no C library at all.

Item 18.28 covers type-driven design: designing types so that the compiler rejects invalid uses of an API instead of relying on documentation to forbid them.