Nullable pointer optimization, callbacks and qsort

Measure the niche optimisation, then hand C a Rust function pointer and let it call you back.

pub fn niche_report() -> Vec<(String, usize)>
pub fn sort_by_abs_desc(v: Vec<i32>) -> Vec<i32>
pub fn sort_pairs(v: Vec<(i32, i32)>) -> Vec<(i32, i32)>

This is genuine C library code sorting a genuine Rust array, calling a genuine Rust comparator through the genuine C ABI — no simulation anywhere.

Part (a): why Option is sometimes free

C says “optional pointer” by using NULL. Rust says it with Option<T>. Those look like completely different mechanisms — one is a magic value, the other is a tagged union with a discriminant — and yet:

Nine of those are free. One is not, and the exception is the whole point.

The compiler performs a niche optimisation: if a type has a bit pattern that can never be a valid value, Option uses that pattern for None instead of adding a discriminant. A reference can never be null. A Box can never be null. A function pointer can never be null. NonZeroU32 cannot be zero — that is literally its name. Each of those has a spare “niche”, so Option costs nothing.

A raw pointer legitimately may be null. That is what raw pointers are for. There is no spare value, so Option<*const u8> has to add a discriminant, pay alignment padding, and double in size.

Which gives you the precise, measurable rule for callbacks:

Option<extern "C" fn(...)> is the correct type for a nullable C callback, and it is layout-identical to the bare function pointer. Option<*const T> must never cross a boundary.

Not a style guideline — an ABI fact you can print. And it resolves the loose end from 18.25, where Option<T> appeared in the “not FFI-safe” column: it is FFI-safe exactly when the niche optimisation applies.

Part (b): qsort

unsafe extern "C" {
    fn qsort(
        base: *mut c_void,
        nmemb: usize,
        size: usize,
        compar: unsafe extern "C" fn(*const c_void, *const c_void) -> c_int,
    );
}

qsort is not generic. It has no idea what it is sorting. You tell it the base address, how many elements, how big each element is, and a function that compares two of them by address.

Note the comparator’s type: it is unsafe extern "C" fn, not a closure. C function pointers carry no environment. A Rust closure that captures anything is a struct with a call method, and there is nowhere in a bare function pointer to put the captured data. A non-capturing closure does coerce to fn, but the moment you need context you must pass it explicitly — which is what a void* context parameter is for, and what item 18.27 covers.

Three ways this goes wrong:

Panicking in the callback. The panic unwinds out of a Rust frame into a C frame, and under the default extern "C" ABI that aborts the process (18.21). Your comparator must be panic-free. No indexing that could go out of range, no unwrap, no arithmetic that could overflow in a build with checks on.

Subtracting. The classic C comparator is return *a - *b;. In C that overflows for large values; in Rust with -O (overflow checks off, which is how this is compiled) it wraps silently, and 2000000000 - (-2000000000) comes out negative. Use cmp and cast the Ordering:

y.unsigned_abs().cmp(&x.unsigned_abs()).then(x.cmp(&y)) as c_int

Ordering is #[repr(i8)] with Less = -1, Equal = 0, Greater = 1, so as c_int gives exactly the -1/0/1 that qsort wants. Clippy is fussy here in a useful way: a clean b.cmp(&a) as c_int passes, while a convoluted comparator with redundant casts trips unnecessary_cast. The gate rewards the clean formulation.

Note unsigned_abs() rather than abs(). i32::MIN.abs() has no representable answer — it panics with checks on and returns i32::MIN with them off, which would make your comparator inconsistent. unsigned_abs() returns u32 and is total.

An inconsistent comparator. If your comparison is not a strict weak ordering — if it can say a < b and b < a, or is not transitive — the C standard says the behaviour of qsort is undefined. Not “you get a badly sorted array”: undefined. Real implementations can read past the end of the array.

That is why the comparator here has a tie-break. Ordering only by absolute value leaves -5 and 5 mutually equal, and qsort is not stable, so their relative order would be unspecified — non-deterministic output. .then(x.cmp(&y)) makes the order total, which makes the result reproducible.

sort_pairs and #[repr(C)]

A Rust tuple (i32, i32) is not FFI-safe (18.25) — its field order is unspecified. To sort pairs you need a #[repr(C)] struct, which fixes the layout so that reading *x.cast::<Pair>() inside the comparator is well-defined:

#[repr(C)]
#[derive(Clone, Copy)]
struct Pair { a: i32, b: i32 }

Then size_of::<Pair>() is what you hand qsort as the element size. Get that number wrong and qsort strides through the array at the wrong pitch, shuffling half-elements into each other.

::: question sort_by_abs_desc guards with if v.len() > 1. Is that necessary, or just tidy?

Tidy, and worth doing anyway.

qsort with nmemb == 0 is well-defined and does nothing, and v.as_mut_ptr() on an empty Vec returns a dangling-but-aligned pointer, which is fine to pass so long as nothing dereferences it. So the guard is not load-bearing.

It is still good practice, for a reason that generalises to all FFI: the set of inputs a C function accepts is defined by prose in a standard, and edge cases like “zero elements” and “null base pointer” are exactly where implementations have historically disagreed. Handling the degenerate case on the Rust side costs one branch and removes the need to be sure about somebody else’s edge-case behaviour.

You saw the same instinct in 18.24, where rust_sum checks for null before building a slice. :::

Your job

Fill in the ten sizes in niche_report — predict each one before you run it — and fix the two comparators. The starter sorts by ascending magnitude with no tie-break, and orders pairs by the wrong field.