Why a good guesser and a good compressor are the same keeper. · Source

Two lighthouses, one strangely short conversation

Two lighthouses face each other across a black strait, and every night their keepers talk in flashes of lamplight. A visitor starts timing them: the two old keepers settle a whole storm's plans in a handful of flashes, while the new man on the south light labors past midnight to say anything. Same lamps, same sea. What makes a conversation short?

An economy built on exhausting light

Flashing is work — hauled shutters, burning oil, arms aching in the cold. So over the years the two old keepers struck a private economy: whatever they can both see coming gets the briefest pattern, and the rare alarms keep the long ones. Every message they might ever send has its length. But how short does each one deserve to be?

Shorter flashes for surer things

\ell(x) \approx -\log_2 p(x)

Their habit obeys a rule neither keeper ever wrote down: a pattern's length should match the odds of the thing it names. An even-odds question deserves a single flash; each halving of the odds earns one flash more. The lamp spends light exactly where certainty runs out. And certainty lives not in the lamp — it lives in the keeper.

Brevity measures how well you finish my thought

Watch the north keeper mid-message: he already sees the three ways the sentence could end, so he waits only for the flash that picks among those. The better each keeper finishes the other's thought, the fewer flashes settle it. The visitor writes down one strange finding: their brevity is a measurement of prediction. Then the south keeper retires.

The new keeper pays full price for every surprise

The replacement predicts nothing. Every tide report startles him, so everything must be spelled out the long way, and the nightly toll of flashes balloons. But season by season his guesses sharpen — and the conversations shorten by exactly that much. His whole progress can be read off the lamp: fewer flashes, better prophet. The trade has a name.

Prediction is compression — one skill, two coats

\mathcal{L} = -\tfrac{1}{T}\sum_{t=1}^{T} \log_2 q(x_t \mid x_{<t})

Prediction is compression. A mind that guesses the next signal well can name whole messages in a few flashes — and a language model is trained by shrinking its average surprise at the next word, which is literally the length of the code it would spend on the world's text. Every falling loss curve is a conversation growing shorter.

🌱 Is the uncompressible just the truly new?

Years on, the two old keepers' nights fell almost silent — near-perfect prediction needs almost no light. Only what neither saw coming still cost full price: the rogue wave, the unseasonal ship. 🌱 If knowing someone means needing fewer words with them, is the part of the sea no keeper can ever shorten just another name for the new?