Skip to content
← All Rust articles

Rust explainer

Enum layout: niches, guarantees and measurements

About 5 min to read. Allow extra time to work through examples.

Learn how unused representations can encode enum variants, which Option layouts are guaranteed, and how to measure the rest without assuming a portable format.

This article uses Rust. You can make it your practice mode to browse related content across the site.

An enum must distinguish its variants, but that does not always require a separate tag field. Sometimes a payload type already leaves room for that distinction in representations it cannot use.

A niche is an unused representation

Every bit pattern of a u32 represents a valid number. NonZeroU32 excludes zero. That unused representation can distinguish None from a present nonzero value in Option<NonZeroU32>.

A non-null reference similarly leaves a representation available for an optional reference. This is called a niche optimization. The resulting enum still has a logical discriminant, even when no separate tag is stored.

A nonzero address alone is not enough to make a reference valid. Alignment, lifetime and other reference requirements still apply. Niche optimization does not make arbitrary pointer bits safe to reinterpret as a reference.

Start with documented guarantees

The standard library documents size, alignment and calling-ABI equivalence between Option<T> and specific payload types. Concrete examples include Box<i32>, &i32, &mut i32, function pointers, NonNull<i32> and NonZeroU32. Qualifying transparent wrappers inherit the documented behavior.

Consult the Option representation table for its exact conditions. The all-zero encoding of None is an additional guarantee with its own restrictions, including sized pointees for the pointer cases. Do not extend that byte-level claim to every unsized or allocator-parameterized type.

These assertions use documented relationships instead of a machine's pointer width:

use std::mem::{align_of, size_of};
use std::num::NonZeroU32;

fn main() {
    assert_eq!(size_of::<Option<Box<i32>>>(), size_of::<Box<i32>>());
    assert_eq!(align_of::<Option<Box<i32>>>(), align_of::<Box<i32>>());
    assert_eq!(size_of::<Option<NonZeroU32>>(), size_of::<NonZeroU32>());
    assert_eq!(size_of::<NonZeroU32>(), size_of::<u32>());
    assert!(NonZeroU32::new(0).is_none());
    assert_eq!(NonZeroU32::new(7).unwrap().get(), 7);
}

The nonzero type expresses a value invariant as well as enabling a compact optional representation. Use it when that invariant fits the domain.

Measure layouts that are not promised

Default Rust representation has alignment and validity rules, but it does not fix all field ordering, tag placement or enum-size choices. A size relationship can be implementation-dependent just as an exact byte count can.

Run this program and record the compiler version and target with its output:

use std::error::Error;
use std::mem::{align_of, size_of};

fn show<T>(name: &str) {
    println!("{name}: size={}, align={}", size_of::<T>(), align_of::<T>());
}

fn main() {
    show::<i32>("i32");
    show::<Option<i32>>("Option<i32>");
    show::<Option<bool>>("Option<bool>");
    show::<Option<Option<bool>>>("Option<Option<bool>>");
    show::<Option<char>>("Option<char>");
    show::<Vec<u8>>("Vec<u8>");
    show::<Option<Vec<u8>>>("Option<Vec<u8>>");
    show::<Box<dyn Error>>("Box<dyn Error>");
    show::<Result<(), Box<dyn Error>>>("Result<(), Box<dyn Error>>");
}

Option<i32> needs to represent more states than i32, whose bit patterns are already occupied. Do not infer its exact tag size or field arrangement from the total size alone. A bool or char leaves invalid representations that a compiler can use for niches, but a measured compact layout is not automatically a stable guarantee for every nested enum.

Box<dyn Error> includes trait-object metadata as well as a data pointer. A result using it must not be described as one ordinary pointer wide merely because a niche can distinguish the empty variant. Optional containers may also have compact layouts on a particular toolchain; check the documentation before relying on those observations in unsafe code or interfaces.

Inline size is not total cost

size_of::<T>() reports the inline size, including padding, used for a value of T in an array. It does not include heap allocations owned by that value. Equal sizes for Option<Box<i32>> and Box<i32> say nothing by themselves about allocation time, branching, dereferencing or cache behavior.

Every value of an enum type has the same size regardless of its active variant. That size must accommodate its payloads and any required tag and padding; it is not necessarily exactly the size of the largest payload. Boxing a large variant can reduce inline storage at the cost of indirection and allocation. Measure those tradeoffs in the workload that matters.

Representation attributes serve specific contracts

repr(C) specifies a target-dependent layout compatible with the relevant C representation rules. repr(transparent) provides the layout and ABI of its qualifying nontrivial field. An integer representation on an enum fixes its discriminant representation within the applicable layout rules.

None of these attributes makes raw memory a portable serialization format. Endianness, padding, pointers, valid enum values and target-dependent sizes still need explicit handling. For files and network messages, define and encode a format rather than writing arbitrary struct memory.

Check your understanding Should an internal struct use repr(C) because explicit layout is faster? Show answer

The attribute constrains layout; it does not guarantee a speedup or slowdown. It can increase padding by fixing declaration order, and layout can affect cache behavior in workload-specific ways. Use a representation required by the interface, or justify an internal choice with measurements and a clear maintenance reason. Default Rust layout is not a promise of globally minimal size either.

Choose types that express the intended states first. Use documented layout guarantees when correctness depends on representation, and measurements when investigating size or performance. Keep those two kinds of evidence separate.

step 3 of 26

Smart Pointers and Interior Mutability

Up next Reduce an enum’s inline size with Box