Same tokens, different state
Two sibling rollouts emit the same 15-token span, yet already read apart on named axes. The trace under-determines the process; the readout does not.
Fudan University · Shanghai Innovation Institute · arXiv:2608.17638
What is the model thinking that it never writes down?
JAR — a 64-axis readout of latent reasoning state, and the native expert routing that reconstructs it for free
JAR·hidden states become a dashboard; routing makes it deployable
A trace records what a model said. A readout shows what it was holding.
The one idea
The trace is what the model chose to emit — not the state that produced it.
A reasoning trace does not say which constraints are still active, whether a long derivation reflects allotted effort or genuine difficulty, or whether the current branch has stopped being productive. Those distinctions matter earlier than the final answer can reveal them. JAR reads them off the hidden states directly: a compact, named coordinate system for latent reasoning state — and then shows that the model's own expert routing already carries most of it.
64 axes, no labels
J64 is built from the model's own states — never from outcome, effort or difficulty labels. Names are assigned after construction and change no number.
Routing gets it for free
Reading hidden states costs a forward pass. Expert routing is written out by the generation stack anyway — and reconstructs the same axes.
How JAR works
Distil a vocabulary-scale readout into 64 axes — then learn it from routing.
The Jacobian lens maps an intermediate hidden state onto vocabulary-aligned directions: what the state could put into words, whether or not it does. That readout has one coordinate per vocabulary item — far too large and too redundant to monitor a trajectory with. JAR compresses it to a 64-axis frame, then fits a second, cheaper route into the same coordinates.
- lens
Read the state against the vocabulary
A per-source-layer Jacobian lens carries the residual stream at layer L into the final-layer basis — a fitted lens, not a bare unembedding. A high reading for constraint does not mean the model wrote the word; it means the state supports a verbalizable representation of it.
- 64
Merge near-duplicate directions into axes
Sample states from replayed rollouts, keep the vocabulary directions that receive consistently high readings, and greedily group them at cosine 0.7 into 64 families. Each axis is its family's weighted mean direction. No step consults an outcome, effort or difficulty label, and no axis is hand-picked or edited.
- R64
Fit routing onto the same axes
The generation stack already records expert assignments and gate weights. A single closed-form ridge maps that usage spectrum onto the 64 readings — question-held-out folds, one uniform regularizer. The result, R64, needs no hidden states at deployment.
J64 is a coordinate system for latent reasoning state, not a correctness detector. Axis names are labels on a frame that was fixed before any name was chosen; a handful of axes are position markers on the manifold and are deliberately never read literally.
In the order you would ask them
A readout has to clear five bars, in order.
Each one is worthless without the one before it. A readout that sees nothing new is a decoration; one that cannot be produced cheaply never ships; one that changes no decision is a dashboard nobody reads; and one whose axes are only correlations is not naming anything at all. What follows takes them in that order.
- OneDoes it see anything the words do not?Effort and difficulty land on different axes, and outcome signal rises by 0.096–0.135 AUC.
- TwoCan you get the reading for nothing?Routing reconstructs it at a median per-axis r of 0.69–0.86, keeping 95–100% of the gain.
- ThreeDoes it choose a better answer?+5.2 over a random pick — and +6.4 on questions it was never built from.
- FourCan it act before the run is over?+1.1–5.9 points over a control that keeps everything but branch identity.
- FiveIs any of it real?Edit the experts an axis names and the model does the named thing — a sham does not.
One · what it sees
Does the readout see anything the words do not?
A long derivation has two possible explanations, and the trace cannot separate them: the model was told to think hard, or the problem is hard. J64 puts those on different axes — and then adds outcome signal on top of everything the text already carries.
Effort has a coordinate
The posture axis tracks the allotted budget almost monotonically — question means at −0.51, −0.02 and +0.53 for Low, Medium and High. Nothing about the effort setting was used to build the frame.
Difficulty has a different one
Strain is not posture. Within each effort setting, accuracy falls as strain rises — −0.35, −0.28, −0.32. A run that feels harder to the model is likelier to be wrong, whatever it was told to spend.
Signal the text lacks
Against a baseline reading the same rollout as token occupancy and aggregating it identically, held-out outcome AUC rises 0.584→0.718, 0.586→0.715, 0.730→0.827 — gains of 0.096–0.135.
And it is not that the words are merely hard to read: for 31 of the 64 axes, the family words defining the axis are never emitted at all in the rollout — while the reading keeps moving.
Two · what it costs
Can the model's own routing stand in for the readout?
Reading hidden states costs a forward pass and a place to put the plumbing. Expert routing costs nothing — the generation stack records which experts fired anyway. One closed-form ridge, question-held-out folds, a single uniform regularizer, and the routing spectrum lands in the same 64 coordinates.
How faithful the copy is
Median per-axis correlation is 0.69–0.86 across three models and two families. At the low end — gpt-oss-20b — 30 of 64 axes still clear 0.7, against 0.00 for the same routing shuffled, which is what rules out the proxy quietly reading the question rather than the run. Instrumentation costs sit inside run-to-run noise.
How much of the gain survives
On gpt-oss-20b the proxy preserves 95–100% of J64's outcome increment while reading no hidden states at all. The expensive instrument is what establishes the signal; the free one is what carries it into deployment.
Three · what it chooses
Does it pick a better rollout from a finished pool?
The first place a readout can pay for itself is the easiest to check: sixty-four completed rollouts, one to keep. The selector is fitted on one source benchmark and then frozen — one shared regularizer across every arm, setting and target, never tuned per cell.
+5.2points over a random pick · averaged over five model-settings
+6.4on questions disjoint from frame construction — larger, not smaller
7/8settings where R64-weighted voting beats plain majority voting
The construction-disjoint margin is the one that matters. If the gain came from the frame having memorised its construction pool, the advantage would shrink on questions it never saw. It grows — which is what rules that explanation out.
Both arms improve selection: J64 from hidden states, and R64 from routing alone. The cheap readout is not a degraded copy at decision time — it is a working selector.
Four · when it acts
Can it act while the run is still being written?
Selecting among finished rollouts still pays for all of them. The harder question is whether the readout can tell, part-way through, that this trajectory is not going to work — early enough for stopping it to be worth anything.
The controller
A cumulative-sum controller watches rolling 256-token windows and cuts when the score's running excess crosses a threshold, then resamples. Every operating point is fixed on training questions alone and then frozen — configurations are executed on held-out questions with no maximum taken, so the absolute levels are not optimistic.
The control it beats
The comparison that matters is permSib: the same score sequences permuted between sibling rollouts of the same question. It preserves the marginal distribution, the autocorrelation and everything question-level, and destroys only branch-level information. Against it J64 gains 1.1–5.9 points, and the routing-only proxy retains 0.9–3.2 of them.
Five · whether any of it is real
Does an axis name a mechanism you can act on?
Everything so far is prediction, and prediction is compatible with an axis being a lucky correlation. The test that separates them is intervention: rank experts by how much they write toward a diagnosed axis, edit router logits only, and check the behaviour against a sham edit of equal strength aimed at unrelated experts.
Amplify the diagnosed failure
Non-terminating trajectories load on the case-splitting bundle. Raising the logits of experts that advance it drives accuracy 0.381 → 0.000 and sends 0.98 of runs to the generation limit. The equal-strength sham is just as damaging to accuracy — but only the targeted edit produces the predicted behaviour: sustained “Case n” enumeration, reaching 100 headers in one run, against none in 160 sham runs.
Suppress the diagnosed stall
A compact expert group is overused during the middle fifth of incorrect trajectories (0.085 vs 0.021). Read through J64 it corresponds to a state that keeps restating the problem's requirements instead of executing the operation they call for. Suppressing it shortens generations by 45.9 tokens where the sham lengthens them by 47.6 — and on the focal tetrahedron problem it replaces early numerical guessing with an exact symbolic derivation, recovering the correct answer.
The sham is the point. An edit that merely breaks the model would move accuracy too — it would not produce enumeration when the readout says case-splitting, or symbolic execution when it says the model is stuck restating requirements.
An instrument, not an oracle
J64 is a data-dependent representation: it is built without outcome labels, but it is built from data. Everything fitted on top of it — selectors, vote weights, the prefix score, the reconstruction map — is supervised and trained on source questions only. The frame consumes 24 of the 120 evaluation questions, so construction-disjoint controls are reported alongside the headline numbers rather than instead of them.
Built from 23,040 rollouts · 64 per question · 4 competition sets + GPQA AIME-24 · AIME-25 · BRUMO-25 · HMMT-25 · GPQA Diamond — gpt-oss-20b, gpt-oss-120b, Qwen3-30B-A3B Thinking & Instruct
Watch
What the trace hides, and what routing gives back — a visual essay
Cite
Reference
@misc{chen2026jar,
title = {Beyond the Trace: Coupling an Interpretable Reasoning-State
Readout to Native MoE Routing},
author = {Chen, Kang and Zhao, Sihan and Cao, Yixin and Jiang, Yu-Gang},
year = {2026},
eprint = {2608.17638},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
note = {JAR --- Jacobian Axis Readout},
url = {https://arxiv.org/abs/2608.17638},
} Read the state, not just the sentence.