How a predictor becomes an agent. · Source

Alone, it only writes the next word. In a loop, it acts.

A language model can't do anything. It can't check a price, run a sum, or open a file — it only predicts the next word and stops. Like a glove on a table: inert on its own. But slip it into a loop — let it act, see what happened, and act again — and the predictor becomes a doer. That loop is the whole idea behind an agent.

Its knowledge froze the day training ended.

The weights stopped changing when training finished. So the model has no senses and no hands: ask today's weather and it can only guess; ask it to run the numbers and it can only pretend. Like a ship in a bottle: a whole world rendered in perfect detail, sealed in glass — unable to touch the sea it shows. To be useful now, it needs a way to reach past the glass.

The fix is a loop: think, act, look, repeat.

a_t \sim \pi_\theta(\cdot \mid c_t),\quad o_t = \mathrm{env}(a_t),\quad c_{t+1} = c_t \oplus a_t \oplus o_t

Give it a cycle: reason about what to do, take one action, read back what the world returned as an observation, then reason again. This is ReAct. In plain words: the model proposes a move from everything written so far, the world answers, and that answer is added to the page it reads next. Like fitting a joint: a carpenter cuts, offers the pieces up to see the gap, shaves, and offers again — closing on a tight fit through feedback, not one lucky guess.

It never truly acts. It writes a request — we run it.

\Delta\theta = 0,\qquad a_t \sim p_\theta(\cdot \mid x_{<t}),\qquad o_t\ \text{clamped, not sampled}

The model can't reach outside itself. It just writes a request — search this, compute that — as ordinary words. A harness runs it and pastes the result back into the transcript. Like a pneumatic message tube: you write a note, it whisks off, an answer whisks back — and you never leave your desk. In plain words: nothing is trained mid-loop; the same frozen model keeps predicting the next token, only now the tool's replies are handed to it, not invented.

Thinking alone invents. Acting alone is blind.

p(\text{answer}\mid q)=\sum_{\tau} p(\text{answer}\mid q,\tau)\,p(\tau\mid q),\quad \tau=(a_1,o_1,\dots,a_T,o_T)

Reasoning with no feedback drifts into confident fiction; acting with no thought just flails. Interleave them: each thought picks the next action, and each real observation corrects the next thought. Like tacking upwind: you can't sail straight at the mark and you can't just drift — you zig-zag, reading the wind each leg and re-trimming, and the goal arrives. In plain words: the answer is built over the path the agent actually walked, and because every observation is real, a wrong step is caught, not compounded.

It stops when it says 'done' — or when the budget runs out.

T = \min\bigl(B,\ \min\{\, t : a_t = \texttt{STOP} \,\}\bigr)

Most loops halt themselves: the model stops calling tools and writes a final answer. But not always — it can fixate, repeating one failed move, or trust a wrong observation and spiral. Like a wheel stuck in mud: it churns the same spin forever unless something calls a halt. So we cap it with a step budget B. In plain words: stop the first turn it says 'done,' or at turn B, whichever comes first — B is the leash that guarantees the loop ends.

A frozen predictor, wrapped in a loop, becomes an agent.

Nothing inside the model changed. Yet a next-word predictor, given tools and a loop, can now plan, act, check, and correct its way to a goal it was never handed in one piece. Like a sheepdog working a flock: the dog reads the sheep, darts, reads again, adjusts — and the whole flock funnels to the pen. The intelligence isn't only in the weights. It's in the loop wrapped around them.

🌱 We built the loop. So whose work is the answer?

The thinking never changed — it's the same next word, predicted again and again. All we added was the loop: the tools, the turns, the chance to look back. So when the agent finally does something real, where does the agency live — in the model, or in the loop of feedback we wrapped around it?