What the trace never writes down

Two rollouts of the same model on the same problem emit an identical span, reading "So we need to count assignments of colors to rows and columns such that", and then diverge. One continues "each row shares color with at least one" and gets the answer right; the other continues "the number of white rows is w_r and" and gets it wrong. A bar chart of the J64 reading over that shared span shows the two already separated, at minus 0.1 against plus 2.5 on the specifics axis and plus 0.6 against minus 1.2 on the constraint axis.
Both runs write the same words. The readout underneath them does not agree — and the run reading high on specifics is the one that goes wrong. Fig. 1a, JAR

Two runs of the same model, on the same question, write an identical fifteen-token span — the one in the box above. Word for word, the same. Then one of them goes on to get the answer right and the other goes on to get it wrong.

Look inside the model at the moment that shared span begins, and the two runs are already far apart. That is the whole problem in one picture. We judge a reasoning run by reading what it wrote, because the text is the only thing it hands us. But writing is a choice — a model puts some of what it is holding into words and drops the rest. Whatever separated those two runs was never written down.

The question a trace cannot answer

Here is a version of this that costs real money. A model produces a very long derivation. There are two explanations:

  • it was told to think hard (many reasoning models take an effort setting, low to high), or
  • the problem is hard.

The trace looks the same either way: long. But the two call for opposite responses. In the first case the length is just budget being spent and you should let it run. In the second the model is struggling and you would rather stop and spend that budget somewhere else. You cannot tell which from the words — and you certainly cannot tell early, which is the only time it would be worth knowing.

The paper’s answer is to stop reading the output and start reading the machine. What follows is how you do that, why it turns out to be nearly free, and how you check that you are reading anything real.

Sixty-four dials

While a model reads and writes, it carries a big bundle of numbers that gets updated at every token — call it the working state. Everything the model is tracking lives in there, whether or not any of it comes out as text.

The bundle is not readable on its own. But there is a way to ask it questions. The last thing a language model does, every token, is turn its state into a score for every word in its vocabulary — that is how it picks what to say next. Refit that same step so it works part-way through the computation, point it at a state from the middle, and you have asked: if you had to put yourself into words right now, which words?

What comes back is a score for every word — a hundred thousand numbers per token. True, and useless. So group the words that always move together. What is left is a panel of 64 dials, which the paper calls J64.

Each dial takes its name from the words inside it — caution, arithmetic, drift, constraint — but the naming happens after the dials are fixed. The names are labels stuck on a panel that was already built; they change no number.

And nothing about whether a run was correct, how much effort it was given, or how hard the question was goes into building the dials. That constraint matters more than it sounds. A dial trained to predict correctness could only ever tell you what you already taught it. These were built blind, which is why it means something when they turn out to line up with anything at all.

What the dials show

Remember the long derivation with two explanations. They turn out to sit on different dials.

One of them tracks how hard the model was told to think: turn the effort setting from low to medium to high and that dial’s reading climbs, in order, every time. A different dial tracks how hard the problem feels — and within any single effort setting, the runs that read as more strained are the ones that go on to be wrong. Same budget, different strain, different odds.

So the ambiguity the text could not resolve was never really an ambiguity. It was two different things that the text flattens into one symptom: length.

That is the part I find hardest to argue with, and it generalises. A dial does not measure whether the model wrote its words — it measures whether the state would support them. So a dial can be busy while its words are nowhere in sight. For 31 of the 64, that is exactly what happens: their words never appear in the run at all, and they move the whole way through anyway.

Worth saying that again, because it is easy to skim past. This is not a signal the text makes hard to read. It is a signal the text does not contain.

A trace is what the model said. The dials are what it was holding.

Getting the reading for free

There is a catch, and it is the kind that kills good ideas. Reading a model’s working state means running the model and reaching inside it — an extra pass, extra plumbing, nothing anyone wants in production.

But some models already write part of it down for nothing. In a Mixture-of-Experts model, each layer holds many small sub-networks — the “experts” — and a router picks a handful of them for every single token. Which ones it picked is not something you have to go fishing for. It is a log the system produces while it samples, because it cannot run without making the choice.

So: fit that routing log onto the same 64 dials, with one linear regression. It works. The free copy — the paper calls it R64 — tracks the real one, dial by dial, at a correlation of 0.69 to 0.86. In practice that means it keeps 95–100% of the predictive value while reading no internal state whatsoever.

The check that makes this believable is cheap and brutal: shuffle the routing log first, then refit. The correlation goes to 0.00. So the fit is genuinely reading the routing, not quietly reading the shape of the question through it.

One caveat about scope. The dials themselves work on any model you can look inside; it is the free version that needs a Mixture-of-Experts model, because that is what makes the log exist at all. Which happens to describe most of the frontier reasoning models people are actually running.

What it’s for

Three things, and they are the point of the whole exercise:

  • Pick. Given 64 finished runs, choose the one to keep. Worth about 5 points of accuracy over a random pick — and more, not less, on questions the dials were never built from, which is how you know the panel did not simply memorise the problems it was assembled on.
  • Stop. Watch a run as it is being written, cut it when the reading turns bad, and start another. Worth 1 to 6 points over a control that keeps every statistical property of the score except which run it came from.
  • Steer. Push the router toward the experts that drive one dial, and the model’s reasoning changes in the direction that dial is named after.

Does the dial name a real thing?

The first two of those are prediction, and prediction is compatible with a dial being a lucky correlation. A number that reliably goes up before failures does not prove that the thing it measures is about what its label says.

The test that separates them is intervention. Take the dial for case-splitting — the habit of breaking a problem into “Case 1, Case 2, …” and working through them. If that dial names a real mechanism, then pushing the router toward the experts that drive it should make the model split into cases. It does: a hundred “Case n” headers in a single run.

Accuracy also collapses. On its own that proves nothing, because accuracy collapses just as hard under a sham edit — same strength, aimed at unrelated experts. But the sham produces zero such headers in a hundred and sixty runs. Breaking a model is easy. Making it do the one specific thing a dial is named after is not.

But isn’t this just a probe?

Fair question — people have been training classifiers on hidden states for years. Four things make this one different, and the last two are the ones I would not have expected to work.

It is built without labels, so it cannot be a memorised map from states to answers. It is a coordinate system rather than a single detector, so it says what kind of trouble a run is in and not merely that trouble is coming. It is reproducible from a free signal, which is the difference between a paper and a thing you can run. And it survives intervention — the axes name mechanisms you can push on, which no probe accuracy number can establish.

What it is not is an oracle. The panel is built from data, and everything fitted on top of it — the selectors, the stopping rule, the routing map — is supervised, trained on one set of questions and then frozen. The claim is that it is a useful instrument, not that it knows the answer.

The takeaway

All of this is measured over 23,040 runs — four competition maths sets plus GPQA, across four models from two families — with every operating point fixed on training questions and then frozen.

A reasoning trace is a summary the model wrote for us, and summaries leave things out. The state underneath is richer, and it is legible earlier — before the answer, before the failure, and sometimes about things the model never says at all.

The part I keep coming back to is the second half. You might expect a signal that good to require special access, a modified runtime, a research setup. It does not. If a model routes its tokens through experts, it is already writing down a log that carries most of the same picture — for free, in production, today. The instrument was on the whole time. Nobody was reading the dial.

The full version of this is on the JAR paper page, figures and all. For an earlier note on the same underlying observation — that routing carries things the answer text does not — see Why the answer hides in the routes.