Rust object layout

Every value in Rust occupies a fixed number of bytes determined by its type. The compiler decides the size (total bytes), alignment (what byte boundary the value must start on), and field order for each type.

Alignment and padding

CPUs access memory most efficiently when values sit at addresses that are multiples of their alignment. A u32 has alignment 4, so it should start at an address divisible by 4. A struct's alignment is the maximum alignment of its fields. The compiler inserts padding bytes between fields (and after the last field) to satisfy alignment constraints. These bytes exist in memory but carry no data.

#[repr(C)]
struct Padded {
    a: u8,    // 1 byte
    b: u64,   // 8 bytes, align 8
    c: u8,    // 1 byte
}

Layout representations

Enum layout

Enums need to store which variant is active (the discriminant) alongside the variant's data. The compiler uses two strategies:

tagged explicit discriminant byte(s)

In the straightforward approach, the compiler reserves extra bytes for a tag that records which variant is active. The tag size depends on how many variants the enum has: 1 byte covers up to 256, 2 bytes more, and so on.

enum Shape {
    Circle(f64),       // radius
    Rect(f64, f64),    // width, height
    Point,
}

niche optimized zero-cost discrimination

This is the optimization that gives nichy its name. Some types have invalid bit patterns — values they can never hold. These unused patterns are called niches, and the compiler can repurpose them to encode the discriminant, eliminating the tag entirely.

Niche optimization in detail

A niche is a range of bit patterns that a type guarantees it will never use as a valid value. The compiler tracks niches through the type system and exploits them when building enum layouts.

Where niches come from

Niches propagate upward through structs. If a struct contains a bool, the struct exposes those 254 niches to any enum wrapping it.

The classic example: Option<&T>

type Ref = Option<&'static u64>;

Multiple niche values

When a type has more niches than an enum needs, the extras are remaining niches that propagate further outward. bool has 254 niches. Option<bool> uses one (storing None as byte value 2), leaving 253. So Option<Option<bool>> consumes one more and is still just 1 byte. An enum with up to 254 unit variants wrapping a bool can fit in a single byte the same way.

This stacking works as long as niches remain. Types with only a single niche, like &T, support one layer of Option at no cost, but Option<Option<&T>> exhausts the supply and requires a tag.

When niche optimization cannot apply

• No niches available. Types like u32 or f64 use all their bit patterns, so wrapping them in Option requires a tag byte.
• Not enough niches. One variant is stored directly; every additional variant needs a niche. If there aren't enough, the enum falls back to tagged layout.
• #[repr(C)] or #[repr(u8)] enums. Explicit representation attributes force a predictable tagged layout.

Common misunderstandings

Niche optimization is well-known, but its limits are often misunderstood. Here are cases where it might seem like it should help, but doesn't.

Niches can't encode variant data

When the compiler uses a niche to represent a variant, it uses that bit pattern as the discriminant. This only works for variants that carry no data, or whose data fits elsewhere in the enum's payload (rust-lang/rust#46213).

enum Either {
    Left(NonNull<u8>),
    Right(NonNull<u8>),
}

Padding bytes are not niches

Struct padding exists for alignment but is not tracked as niche space. Even when a struct has trailing padding that could physically hold a discriminant, the compiler doesn't repurpose it (rust-lang/rust#70230, rust-lang/rust#109555). Niches come from type-level value constraints ("references are never null") rather than from incidental unused bytes in the memory layout.

type A = Option<WithNiche>;
type B = Option<WithoutNiche>;

struct WithNiche {
    ptr: NonNull<u64>,
    flag: u8,
    // 7 bytes trailing padding
}
struct WithoutNiche {
    ptr: *mut u64,
    flag: u8,
    // 7 bytes trailing padding
}

Niche optimization only applies when it reduces size

Even when an enum's variants contain types with usable niches, the compiler only uses niche encoding when it makes the layout smaller. If the discriminant already fits in alignment padding, niche encoding produces the same size and the compiler picks a tagged layout instead. Whether niche optimization applies can therefore depend on the target triple.

enum Expr {
    Literal(f64),
    BinOp { op: char, lhs: Box<f64>, rhs: Box<f64> },
    Neg(Box<f64>),
}

What the niche system actually tracks

The compiler tracks niches as value ranges that a type guarantees it will never use. This is a type-level property rather than a layout-level one:

• NonNull<T> reserves value 0 (1 niche), regardless of T's alignment.
• bool reserves values 2..=255 (254 niches).
• &T reserves value 0 (1 niche).
• Bit patterns excluded only by alignment are not tracked as niches.
• Padding bytes between fields are not tracked as niches.

To verify whether niche optimization applies to your types, paste them into the visualizer. A niche optimized badge means the compiler found usable niches; a tagged badge means it fell back.

How to verify

nichy reports what rustc decided. If a result surprises you (as niche optimization surprises everyone the first time), you may want to verify it independently.

There are two questions to answer: is the size right? and is nichy displaying the correct niche value that rustc picked? The visualizer is usually right about the size, but the specific niche value can be surprising (why 0x2 for Option<bool>'s None? why 0xFF for some other enum's unit variant?). Both examples below answer both questions.

With a debugger

Build a binary that constructs the value, set a breakpoint on it, and look at the raw bytes in GDB / LLDB:

fn main() {
   let some_ref: Option<&i32> = Some(&42);
   let none_ref: Option<&i32> = None;
   std::hint::black_box(some_ref);
   std::hint::black_box(none_ref);
}

(lldb) x/8b &some_ref
0x7fffffffb058: 0x00 0xe0 0x59 0x55 0x55 0x55 0x00 0x00   // Some(&42): a pointer
(lldb) x/8b &none_ref
0x7fffffffb060: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00   // None: all zeros

The Some case is a random address produced by rustc; the None case is eight zero bytes. Both occupy exactly 8 bytes, with no tag and no padding.

With transmute

A short program proves the layout end-to-end. Transmute the value into a byte array and assert what you expect:

fn main() {
    use std::mem::{size_of, transmute};

    // 1) Option<&u64> is 8 bytes, the same as &u64. None is the null pointer (0x0).
    assert_eq!(size_of::<Option<&u64>>(), size_of::<&u64>());
    let some_ref: Option<&u64> = Some(&42);
    let none_ref: Option<&u64> = None;
    let some_ref_bytes: [u8; 8] = unsafe { transmute(some_ref) };
    let none_ref_bytes: [u8; 8] = unsafe { transmute(none_ref) };
    assert_eq!(none_ref_bytes, [0; 8], "None encodes as the null pointer (0x0)");
    assert_ne!(some_ref_bytes, [0; 8], "Some(&42) is the raw address, not null");

    // 2) Option<bool> is 1 byte. bool only uses 0 and 1, so 2..=255 are free
    //    niches. rustc picks the smallest one (0x2) for None, matching the
    //    `currently: None = 0x2` that nichy's niche-line shows.
    assert_eq!(size_of::<Option<bool>>(), 1);
    let some_true:  u8 = unsafe { transmute::<Option<bool>, _>(Some(true))  };
    let some_false: u8 = unsafe { transmute::<Option<bool>, _>(Some(false)) };
    let none_bool:  u8 = unsafe { transmute::<Option<bool>, _>(None)        };
    assert_eq!(some_false, 0x00);
    assert_eq!(some_true,  0x01);
    assert_eq!(none_bool,  0x02);

    println!("Option<&u64>  Some(&42)   bytes: {:02X?}", some_ref_bytes);
    println!("Option<&u64>  None        bytes: {:02X?}", none_ref_bytes);
    println!("Option<bool>  Some(true)  = 0x{:02X}", some_true);
    println!("Option<bool>  Some(false) = 0x{:02X}", some_false);
    println!("Option<bool>  None        = 0x{:02X}   <- the niche", none_bool);
}

On the soundness of the transmute: the Option<&T> layout is formally guaranteed by the standard library, and the Rustonomicon notes that transmuting a valid value of one type into a value with a valid bit pattern of another type is not undefined behaviour. For u8, every bit pattern is valid.

Further reading

• Type Layout The Rust Reference

  The authoritative spec for how Rust lays out types in memory — repr(Rust), repr(C), primitive representations, and alignment rules.

• Niche Optimizations in Rust 0xAtticus

  Deep dive into how the compiler identifies niches and encodes enum discriminants, traced through LLVM IR and rustc source.

• Data Representation in Rust The Rustonomicon

  Practical guide to repr(Rust), repr(C), and other representations, oriented toward unsafe code and FFI.

• Option Representation std docs

  Documents the guaranteed null-pointer optimization for Option<&T>, Option<Box<T>>, etc.

• Type Sizes The Rust Performance Book

  Actionable advice on shrinking types to improve cache performance — repr(u8) enums, boxed variants, field reordering.

• A Surprising Enum Size Optimization James Fennell

  How the Rust compiler reuses tag bits from nested enums to reduce memory overhead — beyond classic niche optimization.

^* The surrogate range 0xD800..=0xDFFF is also invalid for char, but rustc currently does not use those values as niches.