Superposition: Why a Neuron Is Not a Feature

Open a trained network, pick a neuron, and look at what makes it fire. Sometimes you find something clean. Far more often you find a neuron that responds to French text, to a particular punctuation pattern, and to DNA sequences — a polysemantic unit that looks like a bug in the model or in your method. It is neither.

The working hypothesis is that concepts live in directions in activation space, not in coordinate axes — and a network packs more directions into a space than that space has dimensions. That packing is called superposition.

Why it is possible. In $\mathbb{R}^d$ you can place exactly $d$ mutually orthogonal vectors. Relax the requirement to almost orthogonal — inner products bounded by some small $\epsilon$ instead of exactly zero — and the number you can fit grows exponentially in $d$. A 4,096-dimensional residual stream has room for far more than 4,096 near-orthogonal directions.

Why it is worth it for the network. Features are sparse: on any single input, only a handful of the thousands of concepts a model knows are actually active. If two features rarely fire together, storing them along directions that overlap costs almost nothing in practice, while the extra capacity is enormous. The network trades interference for storage, and sparsity is what makes the trade cheap.

Two things make it payable, and the second is easy to miss. Sparsity makes collisions rare; the nonlinearity after the layer suppresses part of the interference that remains, so a small amount of overlap does not have to show up in the output. That second mechanism is what lets the toy model below hold five features in two dimensions — a space with no room at all for five near-orthogonal directions. It does not find near-orthogonal ones: it arranges the five about 72° apart, with pairwise cosines running up to 0.83, and relies on sparsity and the nonlinearity to keep the overlap from mattering.

What follows for interpretation. A neuron is one coordinate of a space whose meaningful structure is a set of non-axis-aligned directions. Reading meaning off individual neurons is like reading individual bytes of a compressed representation: sometimes a coordinate does carry a recognizable feature, but far more often the meaning is spread across several of them. When a neuron does look clean, that is a fact about how this network happened to align — not a licence to read the next one the same way.

That is the motivation for sparse autoencoders: train an overcomplete dictionary that reconstructs activations as a sparse combination of many more directions than there are neurons, and read the dictionary elements instead of the neurons. Superposition is the reason the dictionary has to be larger than the layer, and sparsity is the constraint that makes the pieces separable enough to be worth reading. What it does not do is pin them down: two dictionaries trained on the same activations can disagree, with features splitting, merging or absorbing one another — so a dictionary element is a useful handle, not a canonical unit.

What it does not say. Superposition does not claim every feature is in superposition — some do get near-dedicated neurons. It does not claim directions are the only structure, only that a great deal of structure is linear — which probing and steering support: features can be decoded from directions in activation space, and pushing an activation along one of them causally changes behaviour. That is much stronger evidence than a correlation. It is not evidence that the response is proportional to the push, or that the direction names a single concept: networks are nonlinear, with thresholds, saturation and interactions, and a direction that works is not automatically one thing.

The practical consequence for anyone debugging a model: "which neuron does X" is usually the wrong question. The tractable questions are which direction corresponds to X, whether X is linearly decodable from a layer, and what happens to behaviour when you move along that direction.

The full picture — the residual stream, the linear representation hypothesis, probing, steering, and what sparse autoencoders do and do not recover — is in Mechanistic Interpretability: Looking Inside the Black Box.