How a model's final vector becomes a word — and how we read its mind. · Source

The model's last move is the one we never watch.

Spend a whole network, and each word ends as one vector — a single point. The final move turns that point into a confident guess over every word it knows. We rarely look. But that move can be read not just at the top, but at every floor on the way up — and that's where a model's thinking becomes something you can actually see.

One vector, fanned out into a score for every word.

z = W_U\,x,\quad W_U \in \mathbb{R}^{V\times d}\qquad z_i = \langle w_i,\, x\rangle = \lVert w_i\rVert\,\lVert x\rVert\cos\theta_i

Take that final vector and multiply it by the unembedding — one big table with a row per word. Out drops a number, a logit, for each word. Each row is itself a direction in the same space, so a word's score is just how hard the vector points at it. Like a compass in a ring of signposts: the marker the needle aims at dead-on wins. Then softmax, as before, turns the scores into the odds.

Often the same table reads words in and names them out.

W_U = E^{\top}\quad\Rightarrow\quad z_i = \langle e_i,\, x\rangle

Where does that table come from? Many models just reuse the input dictionary — the very matrix that turned each word into a vector, now run in reverse to score one. So a word's logit is how much the final vector looks like that word's own embedding. One table, both directions. Like a key that both locks and unlocks: read meaning in, write it back out. (Not every model ties them — some keep two.)

Aim that read-head at every floor, not just the top.

p^{(\ell)} = \mathrm{softmax}\!\left(W_U\,\mathrm{LN}\!\left(x^{(\ell)}\right)\right)

Here's the trick. Every layer writes into the same shared vector, in the same language — so the read-head built for the top works at any floor. Unembed the vector halfway up and you get the model's draft guess at that depth. Like a glass elevator: read the floor through the wall as you rise past it. This peek-at-every-layer is the logit lens.

Run the lens and the answer shows up early.

Watch the drafts and something striking appears: the final word often locks in well before the top. The middle floors settle on it; the upper floors mostly make it surer, not different. The thinking finishes early; the rest is polish. Like an instant photo developing: the picture is readable long before it's fully sharp. (It's the common pattern, not a law — hard tokens fight on to the end.)

The lens cheats a little. So we grind a sharper one.

p^{(\ell)} = \mathrm{softmax}\!\left(W_U\left(A_\ell\,x^{(\ell)} + b_\ell\right)\right)

One catch: the trick assumes every floor already speaks the top floor's language. Early floors don't, so their drafts can read like nonsense. The fix learns a tiny translator per floor — a small rotate-and-shift that rewrites each vector into the final language before reading it. Like the wrong reading glasses, then the right prescription: the same page, suddenly in focus. This is the tuned lens.

The last step isn't a footnote. It's a window.

\ell^{*} = \min\left\{\ell : \max_i\, p^{(\ell)}_i \ge \tau\right\}

So the output projection is more than plumbing. Aim it inward and a model's hidden state turns legible: watch a fact surface, a choice harden, a wrong turn appear. You can even get off early — once a floor is sure enough, stop climbing. Like a glass-bottom boat: read the depths without diving in. The map back out was a window all along.

It knew the word floors ago. So what came after?

If the answer is already legible halfway up, what are the upper floors for? Maybe they aren't finding the word — maybe they're weighing the ones it won't say, hedging, checking, talking themselves down. A mind that decides early and keeps climbing anyway. 🌱 Is the rest of the climb the thought — or the second-guessing?