Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Pin and Why It Has To Exist

Pin is the type in the standard library that, on first inspection, appears to do nothing. It is just a wrapper. It has no runtime behavior, no special memory layout, no fancy generated code. You can read its definition in five minutes and feel like you understand it, and be wrong.

Pin is the most subtle thing in std. The reason is that what Pin does is enforce a property on its contents that the type system has no other way of expressing: the value will not be moved in memory until it is dropped. That property is necessary for the soundness of self-referential structs, which the async machinery generates by the gigaton. Without Pin, async Rust as it exists today would be unsound. With Pin, it is sound but the API has rough edges that we will spend the rest of the chapter exploring.

The problem Pin solves

Rust assumes, by default, that values can be moved freely. If you have a Vec<T> on the stack at one address, the compiler may legally move it to a different address — for example, when you return it from a function, when you push it into a containing collection, when you pass it by value. Moves are, in Rust, just memcpy. The bytes are copied; the source is now garbage and the destination is the new official location.

For most types, this is fine. A Vec<T> carries its data on the heap, and the heap pointer it stores is unaffected by where the Vec’s own bytes live. Moving the Vec is moving a (ptr, len, cap) triple, which doesn’t invalidate ptr.

But what about a struct that contains a pointer into itself?

#![allow(unused)]
fn main() {
struct SelfRef {
    data: String,
    ptr: *const String,
}

impl SelfRef {
    fn new(s: &str) -> Self {
        let mut sr = SelfRef { data: s.to_string(), ptr: std::ptr::null() };
        sr.ptr = &sr.data;
        sr
    }
}
}

This is unsound the moment you move sr. The ptr field still points at the original location of data, which has now been memcpy’d somewhere else. Dereferencing ptr after the move reads stale memory.

You don’t write this kind of code by hand often. But the compiler does, every time you write an async fn that holds a borrow across an await. The state machine that the compiler generates for an async function looks roughly like:

#![allow(unused)]
fn main() {
enum State {
    Start { data: Vec<u8> },
    Awaiting { data: Vec<u8>, borrow_of_data: *const Vec<u8> },
    Done,
}
}

When the function transitions from Start to Awaiting, the borrow_of_data field is set to &data, but data is itself a field of the same enum. The future is self-referential. If you move it after the transition, the borrow becomes a dangling pointer.

The Rust team had two choices. One: forbid borrows across await points. This would have made async Rust nearly unusable, because every helper function that takes &self and is awaited would be illegal. Two: invent a way to mark futures as “do not move me,” and have the executor honor that constraint.

They chose option two. Pin is the marker.

The Pin API

#![allow(unused)]
fn main() {
pub struct Pin<P> {
    pointer: P,
}
}

Pin<P> wraps a pointer-like type P. The pointer is, typically, &mut T or Box<T> or &T. The contract is: if T: !Unpin, then while the Pin exists, the T it points to will not be moved.

That Unpin qualifier is the escape hatch. Unpin is an auto-trait that says “this type doesn’t care about being pinned.” Most types are Unpin because they don’t have any reason to be pinned. Pin<&mut T> where T: Unpin is essentially the same as &mut T — you can move freely through it. The pin is decorative.

For types that are !Unpin, Pin actually does something. You can’t get a &mut T out of a Pin<&mut T> for !Unpin types via safe code. The only ways to do anything with a Pin<&mut T> are:

  1. Use methods that take Pin<&mut Self> instead of &mut self (which is what Future::poll does).
  2. Use Pin::as_mut to reborrow the pin (which gives you another Pin<&mut T>, not a &mut T).
  3. Use unsafe { Pin::get_unchecked_mut() } to escape the pin, with the obligation to never actually move the pointee.

The whole !Unpin plus Pin machinery is designed so that:

  • You can write methods that operate on a pinned value.
  • You cannot write code that moves a pinned value out from under itself.
  • The unsafe get_unchecked_mut exists for when you need to violate this, taking on the proof obligation manually.

The state machines generated by async fn are !Unpin. They have to be — they are self-referential. So you can’t poll them through a &mut Future; you must poll them through a Pin<&mut Future>. This is why Future::poll takes Pin<&mut Self>. The pinning is the soundness guarantee that lets the state machine contain self-references.

Structural pinning and projection

Now we get to the part that is genuinely thorny.

Suppose you have a struct:

#![allow(unused)]
fn main() {
struct Composite {
    a: SomeFuture,
    b: u32,
}
}

You have a Pin<&mut Composite> and you want to call a.poll(cx). To do that, you need a Pin<&mut SomeFuture> from the Pin<&mut Composite>. This is called projection: deriving a pin to a field from a pin to the containing struct.

The question: is this safe?

It depends. If SomeFuture: !Unpin, then projecting a pin to it through Pin<&mut Composite> requires that moving the Composite would also move the SomeFuture. Which it does — a is a field of Composite, so a memcpy of Composite includes a memcpy of a. So if Composite is pinned (won’t be moved), then a is also pinned (also won’t be moved).

This is structural pinning. The pinning of the parent is structurally inherited by the field. Projecting a pin from Pin<&mut Composite> to Pin<&mut SomeFuture> is sound.

But: this is only sound if you also enforce that nobody can write code that moves a out of Composite while Composite is pinned. For example, if you have a method on Composite that takes &mut self and does std::mem::swap(&mut self.a, &mut other_future), that method moves a. If anyone has a structurally-projected pin to a at that moment, you have unsoundness.

The rules for safe structural pinning are:

  1. The struct must not implement Drop in a way that moves any pinned field. (Or, if it does, the Drop impl must take Pin<&mut Self>, which is non-standard and rare.)
  2. The struct must not provide any method that moves a pinned field through &mut self.
  3. The struct’s Unpin impl, if any, must be conditional on all pinned fields being Unpin.

Getting all three of these right manually is annoying and error-prone. You also have to write the projection methods by hand, with unsafe:

#![allow(unused)]
fn main() {
impl Composite {
    fn project_a(self: Pin<&mut Self>) -> Pin<&mut SomeFuture> {
        unsafe { self.map_unchecked_mut(|s| &mut s.a) }
    }
}
}

This is fine if you do it right. It is unsound if you do it wrong. And the failure mode is silent — the code compiles, runs, and corrupts memory under the right interleaving.

This is why the pin-project crate exists.

What pin-project is doing

pin-project is a procedural macro that takes:

#![allow(unused)]
fn main() {
#[pin_project]
struct Composite {
    #[pin]
    a: SomeFuture,
    b: u32,
}
}

and generates safe projection methods, the right Unpin impl, the right Drop enforcement, and a few other guarantees that make structural pinning sound. The #[pin] attribute marks a as pinned; b is treated as movable. The macro emits a project() method that returns a struct of references with the right pinning on each field:

#![allow(unused)]
fn main() {
let proj = composite.project();
proj.a.poll(cx);  // proj.a: Pin<&mut SomeFuture>
let _ = *proj.b;  // proj.b: &mut u32
}

Everything that would otherwise require unsafe is inside the macro, audited once, and used safely thereafter.

You should use pin-project (or its companion pin-project-lite, which is a smaller no-proc-macro alternative) any time you write a future or stream that contains other futures or streams as fields. Hand-writing the projection is acceptable for small one-off cases but is not a habit you want.

The “safe to move” mental model

Here is the mental model that, once internalized, makes Pin make sense.

Most types are safe to move. They satisfy Unpin. You don’t need Pin for them and Pin<&mut T> for T: Unpin is just &mut T with extra syntax.

Some types are unsafe to move after a certain point. Self-referential types are the canonical example. Once a self-referential type has set up its internal pointers, moving it would break those pointers. These types are !Unpin.

The !Unpin types need to be addressable through a pointer that promises not to let them move. That pointer is Pin<P>. The promise is: from the moment you create the Pin<P> until the T inside is dropped, the T will not move.

The !Unpin types need this promise to be transitive. If you have Pin<&mut Composite>, you need to be able to get Pin<&mut SubFuture> for the SubFuture field. This is structural pinning, and the pinning machinery (manual or via pin-project) makes it sound.

Once pinned, a value can be operated on (via methods that take Pin<&mut Self>) but not moved. It can be read, polled, dropped — anything that doesn’t change its address. It cannot be returned by value, swapped, replaced, or otherwise relocated.

That is Pin. It is a way of saying “this value’s address is now part of its identity,” within a type system that otherwise treats addresses as irrelevant.

Where the model breaks

A few sharp edges remain.

Pin<Box<T>> is the workhorse but the docs underplay it. Almost every real use of Pin is Pin<Box<T>> — pinned because moving a heap allocation doesn’t move the pointee, only the pointer. Box::pin is the constructor. If you need a Pin<&mut T> for arbitrary T: !Unpin, the easiest way to get one is Box::pin(t). The cost is one heap allocation. This is fine for most futures. It is not fine for, e.g., generators in tight loops.

Structural pinning is a per-field decision. Some fields of your struct should be pinned (futures), some shouldn’t be (counters, configuration). The #[pin] attribute lets you choose per field. Pinning everything is correct but limits what your methods can do; pinning nothing means you can’t have any !Unpin fields. The right answer depends on the struct.

Drop and Pin interact subtly. A Drop impl that moves pinned fields is unsound. The Drop trait’s drop method takes &mut self, not Pin<&mut Self>, and you cannot change that. So if your struct has !Unpin fields and you implement Drop, you must be careful not to move those fields in drop. pin-project enforces this for you. Hand-rolled code has to enforce it by discipline.

Pin does not prevent reading or modifying — only moving. A Pin<&mut T> can still mutate T through methods that take Pin<&mut Self>. The pin is about address stability, not about immutability. A common confusion among beginners is to think Pin is some kind of advanced Mutex. It is not. It is purely about whether memcpy is allowed.

Why this is the design

A reasonable question: was there a better way? Several alternatives were considered:

  1. A Move trait the user opts in to. Types that opt out of Move cannot be moved. The problem: this is backwards-incompatible with all existing Rust code, which assumes everything is movable. A type that becomes non-movable would break every user that returned it by value or stored it in a Vec.

  2. Pin everything by default and add an Unpin opt-in. Same problem, in reverse. You can’t make every existing type non-movable without breaking the world.

  3. Make Future::poll take &mut self and disallow self-referential futures. Would prevent the async fn desugaring as it exists. Async would have to use a different model — probably stackful coroutines, with all the costs that implies.

  4. Add Pin as a wrapper around pointers, and have !Unpin types opt in. What we have. Adds a new type to the standard library. Doesn’t break anything. Lets futures be self-referential safely. Has rough edges around projection.

The chosen option is the one that least breaks the existing language at the cost of adding a quirky API at the boundary. This is, in retrospect, almost certainly the right call. It is also the source of the meme that Pin is the worst-designed thing in std. Both of these are true.

What you actually do

In practice, you will use Pin in three ways.

As a consumer of futures. When you write let _ = some_future.await;, the compiler handles all the pinning for you. You will never see Pin in your code. This is the case 95% of the time.

As a writer of futures by hand. When you implement Future directly (not via async fn), you have to take Pin<&mut Self> in your poll method. If your future contains other futures as fields, you’ll use pin-project to project pins to them.

As a writer of generic async utilities. Combinators, runtimes, channel implementations. You will be deep in Pin, Unpin, and structural pinning. Read the source of tokio for examples; it is pleasingly readable for a project of its complexity.

For most application-level Rust, you are firmly in case 1. Knowing the model lets you read the error messages when they appear; you don’t have to live in the model day to day.

Sources