Why rare news costs more — and why some messages can't shrink. · Source

The runner who weighs news before naming a price

A runner carries news between two mountain valleys. For years he charged by the word — until one single word sent him running through the night, while a whole page wasn't worth his boots. Now his price list looks strange: he weighs each message before naming a fee. What, exactly, is he weighing?

Expected news is copper, unheard-of news is silver

s(p) = -\log_2 p

His rule: the expected is cheap, the unheard-of is dear. "The pass is open" costs a copper in summer — everyone knew — and silver in midwinter, when nobody did. A sure thing rides free. The fee follows the odds: halve how likely the news is, and the price climbs one fixed step. Then the treasurer asks about a whole season.

A season of news has an average weight

H = -\sum_i p_i \log_2 p_i

The treasurer weighs each kind of news by how often it tends to arrive, then averages the fees — every outcome's price, weighted by its odds of showing up. That one number is the season's fair rate per message. Steady weather averages cheap; wild weather averages dear. The clever villagers still think they can outwit the bill.

The village tries to outwit the bill

They try everything. The commonest news is trimmed to a single short call; rare tidings keep the long phrasings. It works — the bill drops close to the treasurer's number, and in a calm year the runner barely jogs. But push as they might, the bill never falls below that average surprise. Something is holding a floor.

The floor has a name: entropy

The floor is real. The average surprise of a source is its entropy — and no scheme, however cunning, can carry its news for less. A predictable season makes cheap news-carrying; a chaotic season cannot be made cheap by any cleverness. The messages weigh what they weigh — and the mountain is not the only place that learned it.

Some messages simply cannot be shortened

You meet the runner's floor every day. A file shrinks under compression until it stops — what remains is all surprise, no padding. A transcript of fair coin flips won't squeeze at all, while a diary of identical days almost vanishes. Entropy is the size a message refuses to go below. So the price was never in the words at all…

🌱 Whose surprise sets the price of news?

The runner never charged for the weather itself — he charged for the village's ignorance of it. The old shepherd who reads clouds hears the same midwinter news and shrugs: to him it was likely all along. 🌱 If surprise lives in the listener rather than the sky, whose expectations set the true weight of news?