Function-like proc macros and proc-macro hygiene

Item 18.11 showed that macro_rules! protects your variables from a macro’s variables. Procedural macros do not.

That single sentence is the most important practical difference between the two systems, and it explains something you have certainly noticed without knowing why: generated code looks absurdly verbose.

// what a proc macro emits
::core::result::Result::Ok(::core::option::Option::Some(::std::string::String::new()))

// what a human would write
Ok(Some(String::new()))

That is not a style choice. It is defensive necessity.

The Reference’s wording, and what it means

Procedural macros are unhygienic. This means they behave as if the output token stream was simply written inline to the code it’s next to.

“As if written inline at the call site” is exact. Every identifier in the output resolves in the caller’s scope. The macro’s output is affected by, and affects, whatever is in scope where it was invoked.

So this breaks:

struct Option;                       // a user's own type, shadowing the prelude

#[derive(MyDerive)]                  // emits `Option::Some(x)`
struct Thing { ... }

The generated Option::Some now resolves to the user’s Option, which has no Some. The derive is correct; the user’s code is legal; the combination fails with an error pointing at code the user never wrote.

Absolute paths fix it. ::core::option::Option::Some starts at the crate root of core and cannot be shadowed by anything in the user’s scope. Hence the verbosity — every path in generated code is written out in full, ::-prefixed, by authors who have been bitten.

The same reasoning drives the other conventions you see in generated code:

• Long, unlikely identifiers. __self_0, __derive_more_field, _serde::__private. A macro that needs a temporary cannot use x, because the user might have an x and the macro would capture it.
• const _: () = { ... }; wrappers. Serde wraps its generated impls in an anonymous const block so its use statements and helper types cannot leak into the user’s module. It is the closest thing to a scope a proc macro has.
• Trait method calls written as Trait::method(&x) rather than x.method(). Method resolution can be hijacked by an inherent method with the same name; the fully-qualified form cannot.

Partial control: call_site vs mixed_site

Every token a proc macro emits carries a Span, and the span determines where its identifier resolves. Two are available on stable:

• Span::call_site() — resolves at the call site. Fully unhygienic, and the default for anything quote! produces.
• Span::mixed_site() — the macro_rules! behaviour: local variables resolve at the definition site, everything else at the call site (18.11). Use it for identifiers you introduce as temporaries, and they stop colliding.

A third, Span::def_site() — everything resolves at the definition site, full hygiene — remains unstable. It is the one people want and it is not available.

So the practical position is: quote! gives you call_site by default, you can opt individual identifiers into mixed_site, and full hygiene is out of reach. Defensive paths are not a workaround for laziness; they are the state of the art.

Function-like proc macros: the payoff for all this

The reason to accept unhygienic output is that function-like proc macros can accept token soup that is not valid Rust.

sql!(SELECT name, email FROM users WHERE id = 42)
json!({ "items": [1, 2, 3], "ok": true })
html!(<div class="row">{ content }</div>)

None of those are Rust expressions. macro_rules! cannot help, because every fragment specifier it offers describes a piece of Rust grammar (18.3) — there is no :sql and never will be. A proc macro receives a raw TokenStream and is free to parse it as anything that lexes.

That is the whole advantage, and it is a large one. sqlx::query! parses SQL at compile time and checks it against a real database schema; nothing declarative could come close.

The lexer is still a limit. The input must tokenise as Rust, so unbalanced delimiters, stray # outside an attribute position, and unterminated strings are all rejected before your macro ever runs. json!({"a": 1}) works because JSON’s punctuation happens to lex fine; a syntax with <<< heredocs would not.

The token model is not tt

If you go from macro_rules! to writing a proc macro, this will trip you up.

A TokenStream is a flat sequence of four things: Group, Ident, Punct, Literal. And:

• Multi-character operators are split into individual Puncts. => arrives as = then >, :: as : then :, ..= as three. Each Punct carries a spacing field — Joint if it is glued to the next one, Alone otherwise — and reassembling operators from that is your problem. (syn does it for you, which is a large part of why everyone uses syn.)
• A lifetime is ' followed by an Ident. Not one token.
• Only Group has structure. Delimiters are the only nesting.

So the intuition you built about tt — “one token tree, delimiters included” — transfers only for Group. Everything else is finer-grained than you expect.

Why macro_rules! is still the right default

Put the two side by side:

	macro_rules!	procedural
hygiene	mixed-site, protects locals	none
input	must be Rust grammar fragments	any token soup
type info	none	none (still just tokens)
lives in	your crate	a separate crate
build cost	zero	compiles syn + quote
runs arbitrary code at build time	no	yes
IDE support	degraded	worse
debugging	poor	poorer

Note the row that says “type info: none” for both. A proc macro sees more structure than a macro_rules! macro — it can parse the tokens into a real syntax tree and walk the fields — but it still does not know types, does not know trait impls, and cannot ask the compiler anything. #[derive(Serialize)] works by generating code that will later fail to compile if a field is not serialisable, not by checking up front.

Given all that, the ordering from 18.16 holds: generics and traits, then macro_rules!, then a proc macro when you need structure or non-Rust syntax. Item 18.20 makes the decision explicit, including where reasonable people disagree.