Skip to content

← Macros, FFI and Type-Driven Design step 12 of 28

Hard Primitives

Follow-set rules: why the compiler rejects your matcher

Four macro definitions, four different follow-set violations, and not one of them has been called yet.

pub fn all() -> Vec<i64>

The starter fails to compile with errors like:

error: `$a:expr` is followed by `+`, which is not allowed for `expr` fragments
```

These fire when the macro is **written**, not when it is used. `all()` is
correct; the call sites are correct; every one of the errors is in a matcher.

Your job is to repair the four matchers **without touching a single call site**.

## Why the compiler is being like this

A matcher is parsed left to right, and when it reaches `$e:expr` it hands the
remaining tokens to the real expression parser. That parser has to decide where
the expression stops — and the only signal available is the token that comes
next.

If the next token could plausibly continue the expression, the boundary is
ambiguous. `$e:expr +` is the clearest case: given `1 + 2`, should `$e` be `1`
and then match the literal `+`, or should `$e` be the whole `1 + 2`? Today Rust's
grammar might settle it. Next year's grammar might not, and every macro that
relied on today's answer would break.

So the language fixes a conservative **follow set** per fragment kind: the tokens
that are permitted to appear immediately after it in a matcher. Anything else is
rejected at definition time, whether or not your particular grammar is actually
ambiguous. Forward compatibility is bought with a small amount of present-day
inconvenience.

## The table

| after | may be followed by |
| --- | --- |
| `expr`, `stmt` | `=>` `,` `;` |
| `ty`, `path` | `=>` `,` `=` `\|` `;` `:` `>` `>>` `[` `{` `as` `where`, or a `block` fragment |
| `pat` | `=>` `,` `=` `if` `in` |
| `pat_param` | the same, plus `\|` |
| everything else | anything |

`ident`, `literal`, `lifetime`, `tt`, `item`, `meta`, `vis`, `block` are
unrestricted, because each is decided without ambiguity about where it ends.

## The four repairs

**`add_pair!(1 + 2)`.** An operator after `expr` is the flat "no" case; there is
no arrangement of `expr` that survives here. Two ways out: switch the fragments
to a kind with no follow restriction, or bracket them so the matcher does not
need the operator to delimit anything. Since both operands are literals, the
first is a one-word change.

**`type_size!(u32 bytes 4)`.** `bytes` is an identifier, and identifiers are not
in `ty`'s follow set. Same escape: pick an unrestricted fragment kind that can
still be used in type position.

**`matches_either!(2, 1 | 2)`.** This is the interesting one. `pat` used to allow
`|` after it. Then edition 2021 made top-level or-patterns part of `pat` itself,
so `$p:pat` given `1 | 2` would now swallow both alternatives — and `|` had to
leave the follow set to keep that unambiguous. `pat_param` is the fragment kind
that kept the old, narrower meaning: it stops before a top-level `|` and
therefore may legally be followed by one. The repair is one word and the call
site is untouched.

**`run_then!(let z = 3; then z * 2)`.** `then` is not in `stmt`'s follow set —
but `;` is. And recall from 18.3 that `$s:stmt` does not consume the trailing
semicolon, so the `;` in the call site is still sitting there unmatched. Writing
it into the matcher fixes the follow-set violation and the leftover token in one
move.

::: question `run_then!`'s transcriber is `{ $s $e }` with no semicolon after `$s`. Isn't a statement supposed to end with one?

It already does.

`$s` is a `stmt` fragment, which means the token stream carries it as a single
pre-parsed statement node — a complete statement, self-delimiting. Writing `$s;`
appends a second, *empty* statement, and rustc's warn-by-default
`redundant_semicolons` lint says so:

warning: unnecessary trailing semicolon


Under `clippy -D warnings` that is a failed submission. It is a nice illustration
of the difference between the two levels this track keeps moving between: at the
token level the `;` looks necessary, and at the AST level it obviously is not.
:::

## A note on shape

Every repair here is one or two tokens. That is characteristic: follow-set errors
look severe and are almost always a trivially small fix once you know which
column of the table you are in. The message even tells you which fragment is at
fault and which token offended it — the only thing it does not tell you is the
list of tokens that would have been acceptable.