Skip to content

← Unsafe and Soundness step 23 of 24

Hard Primitives

union, transmute and MaybeUninit 

#[repr(C)]
union Payload { i: i64, f: f64, b: [u8; 8] }

pub fn tagged_union_decode(records: Vec<i64>) -> Vec<String>

A union is a type whose fields all share the same storage. Writing one field and then reading another reinterprets the same bytes as a different type. Rust does not record which field was written last, so a program that uses a union usually keeps a separate tag: a value that says how the bytes should be read.

In this exercise, records is a flat list of pairs: [tag0, bits0, tag1, bits1, ...]. For each complete pair, store bits in a Payload through the i field, then produce one output string chosen by the tag:

tag field to read output string
0 .i the i64 in decimal, for example -7
1 .f the f64 formatted with format!("{value:?}")
2 .b sum=N, where N is the sum of the eight bytes
any other value none invalid

Contract details:

  • Return one string per complete pair, in input order. If records has an odd length, ignore the last element. Empty input returns an empty vector.
  • bits can be any i64, including i64::MIN and i64::MAX.
  • For tag 1, the eight bytes of bits are read as an f64. Use Rust's Debug formatting exactly. For example, the bit patterns of 0.0, -0.0, 1.5, positive infinity and a NaN print as 0.0, -0.0, 1.5, inf and NaN. Very large or very small magnitudes use exponent notation, such as 1e300. Every float the tests use prints in plain form: the five above plus 2.0, 0.5 and -2.25.
  • For tag 2, each byte is an unsigned value from 0 to 255, so the sum is between 0 and 2040. Addition does not depend on byte order, so the result is the same on little-endian and big-endian targets. Sum into a type wider than u8.
  • Tags are compared as i64; negative tags such as -1 are invalid.

Check the tag before reading any union field. In this exercise all three fields accept any 8-byte pattern, so no read here produces an invalid value, but the habit matters: in general, reading a field the tag does not describe can produce an invalid value, which is undefined behavior.

The starter does not compile. Reading a union field is one of the operations Rust allows only inside an unsafe block, and rustc reports E0133 for the starter's unguarded reads. The starter also enables clippy::undocumented_unsafe_blocks, so each unsafe block you add needs a // SAFETY: comment explaining why the read is valid.

(a) union: writing is safe, reading is not

Assigning to a union field, as in u.f = 2.0, is safe: it overwrites bytes and cannot by itself produce an invalid value. Reading a field, or taking a reference to one with &u.f or &mut u.f, requires unsafe. The reason:

Reading a union field is undefined behavior unless the bytes currently stored are a valid value of that field's type.

A type's validity invariant is the set of byte patterns that are allowed to exist as a value of that type at all. bool allows only the bytes 0 and 1, so reading a stored 3 through a bool field is undefined behavior. Every initialized 8-byte pattern, by contrast, is a valid i64, a valid f64 (including infinities and NaNs) and a valid [u8; 8]. Because this exercise always writes all eight bytes through .i, any of the three fields can be read afterward.

Three more rules:

Field types are restricted. A union field must be a Copy type, a reference, a ManuallyDrop<T>, or a tuple or array made only of those. A field such as String is rejected with E0740. Rust cannot know which field is currently stored, so it never runs a destructor for a union's contents; this restriction keeps that from silently leaking or skipping cleanup.

Bytes that were never written are uninitialized. Consider union U { small: u8, big: u32 }. After writing only small, three bytes of big were never written. Reading big is undefined behavior even though every initialized 32-bit pattern is a valid u32: uninitialized memory is not "some unknown number", and integer types do not permit it. The article on validity and safety invariants covers this distinction in more detail.

Use #[repr(C)] when layout matters. With the default representation, the Rust Reference does not guarantee that every field starts at offset 0. With #[repr(C)] every field starts at offset 0, so all three fields of Payload cover the same eight bytes. Clippy's default_union_representation lint (off by default) flags unions without an explicit representation.

Check your understanding let a = unsafe { &mut u.b }; let c = unsafe { &mut u.i }; uses two different fields. Does it compile if both references are used afterward? Show answer

No. rustc reports E0499:

error[E0499]: cannot borrow `u` (via `u.i`) as mutable more than once at a time

with the note that u.i is a field of the union, so it overlaps u.b. Borrowing one field of a union borrows all of its fields for the same lifetime. In a struct, &mut s.a and &mut s.b refer to separate fields and can coexist (aliasing XOR mutability).

Union fields occupy the same bytes, so two live &mut references to different fields would be two mutable references to overlapping memory, which Rust's borrowing rules forbid.

(b) transmute, and safer alternatives

std::mem::transmute::<Src, Dst>(value) reinterprets the bytes of a Src value as a Dst. The Rustonomicon describes it as "the most horribly unsafe thing you can do in Rust". The compiler checks only that the two types have the same size and reports E0512 otherwise. Everything else is the caller's responsibility:

  1. The result must be a valid Dst. Producing an invalid value, such as transmuting the byte 3 into a bool, is undefined behavior immediately, even if the value is never used afterward.
  2. The type arguments can be inferred. If you omit ::<Src, Dst>, the surrounding code decides the types, and a change elsewhere can change what is transmuted. Clippy's missing_transmute_annotations lint (warn by default) flags calls whose types are not written at or next to the call.
  3. Transmuting &T into &mut T is always undefined behavior. The compiler may assume that data behind a shared reference does not change. rustc rejects this transmute with the mutable_transmutes lint, which is deny-by-default.
  4. Transmuting to a reference can create an unbounded lifetime. If the result type does not name a lifetime tied to the source, the caller's context chooses it, and the borrow checker can no longer protect the original data.
  5. Matching sizes do not imply matching layouts. Structs and enums with the default representation have no guaranteed field order or padding, so transmuting between them depends on details the compiler may change. Use #[repr(C)] or #[repr(transparent)] when the layout must be relied on.

std::mem::transmute_copy has no compile-time size check at all. It reads size_of::<Dst>() bytes from behind a reference, and calling it with a Dst larger than Src is undefined behavior according to its documentation.

Many common reinterpretations have dedicated APIs:

instead of transmuting use needs unsafe?
f32/f64 to and from raw bits to_bits / from_bits no
integer to and from bytes to_ne_bytes / from_ne_bytes, or the _le_ / _be_ forms no
u32 to char char::from_u32, which returns None for invalid values no
numeric conversions as, From, TryFrom no
raw pointer type to raw pointer type the pointer cast method no (dereferencing the result does)
&[u8] to &[u32] slice::align_to yes

For several of these cases, rustc's unnecessary_transmutes lint (warn by default) points out the safe replacement.

Check your understanding slice::align_to::<u32>() returns three slices instead of one. Why? Show answer

A u32 normally requires 4-byte alignment, but a &[u8] can start at any address. Only part of the byte slice may be reinterpretable as properly aligned u32 values.

let (prefix, middle, suffix) = unsafe { bytes.align_to::<u32>() };

prefix holds leading bytes, middle is an aligned &[u32], and suffix holds trailing bytes that do not fit. For example, if a 10-byte slice starts 1 byte past a 4-byte boundary, the largest possible middle gives parts of 3 bytes, one u32 and 3 bytes. The documentation allows align_to to return a shorter middle than possible, even an empty one, so correct code handles all three parts and must not depend on middle being as long as possible.

A plain transmute::<&[u8], &[u32]>(bytes) does none of this: the result can be misaligned, and the slice length is not converted: on current compilers the length is copied unchanged, so the result claims four times as many bytes as the original slice covers. Both problems are undefined behavior.

align_to is still unsafe because it does not check that the middle bytes form valid values of the target type. For u32, every initialized 4-byte pattern is valid, which is the argument the // SAFETY: comment should state.

(c) MaybeUninit: a value that may not be initialized yet

std::mem::uninitialized::<T>() is deprecated: it claims to return a T while leaving the bytes uninitialized, which is undefined behavior for almost every T, including integers. rustc's invalid_value lint warns on mem::uninitialized::<u32>() with the note "integers must be initialized". std::mem::zeroed::<T>() is different: it is valid only for types where the all-zero byte pattern is allowed. It is fine for integers and floats, but undefined behavior for references, NonZero types and function pointers; invalid_value also warns on zeroing a reference.

MaybeUninit<T> is a union in the standard library, marked #[repr(transparent)], with the same size, alignment and ABI as T. The compiler assumes nothing about its contents until you claim, with an unsafe call, that it holds a valid T.

The main API: uninit, zeroed and new create it; write(val) stores a value and returns &mut T; as_ptr and as_mut_ptr give raw pointers; and the unsafe methods assume_init, assume_init_ref, assume_init_mut, assume_init_read and assume_init_drop treat the contents as an initialized T.

assume_init requires a fully valid T. Every part of the value that T's validity invariant requires must be initialized and valid. Padding bytes between struct fields are not part of the value and need not be initialized, so a struct can be initialized in place by writing each field through a raw pointer to it.

Some practical details:

  • MaybeUninit<T> has no niche (no invalid bit pattern for Option to reuse), so on the current compiler Option<MaybeUninit<bool>> is 2 bytes while Option<bool> is 1.
  • An array of uninitialized elements can be written as [const { MaybeUninit::uninit() }; N]. There is no stable MaybeUninit::uninit_array function.
  • MaybeUninit<T> implements Clone only when T: Copy, so vec![MaybeUninit::<String>::uninit(); n] does not compile (E0277). Use (0..n).map(|_| MaybeUninit::uninit()).collect() instead.

MaybeUninit never drops its contents. If you initialize some elements of an array of MaybeUninit<T>, you are responsible for dropping exactly those elements, for example with assume_init_drop, including when a panic interrupts initialization. The MaybeUninit ring buffer exercise practices this.

Clippy has two deny-by-default lints for common mistakes: uninit_assumed_init (for example MaybeUninit::uninit().assume_init()) and uninit_vec (Vec::with_capacity followed immediately by set_len).

What the tests cannot check

If you build the Payload and read a field before validating the tag, the output can still be correct and every test can still pass: in this exercise each field accepts every initialized byte pattern. Undefined behavior is a property of the program's rules, not something an output comparison can observe. The starter fails because rustc requires unsafe around union reads, not because a test detected undefined behavior.

Passing requires compilation, correct output for every test, and Clippy with warnings treated as errors.