Routing Effective Rank — When Do Reasoning Chains Diverge?

The one idea

A cohort is not a bag of answers. It has a shape — and the shape moves.

Test-time scaling turns inference into a population process: a model doesn't return one answer, it returns a cohort of reasoning rollouts that a vote or a verifier later collapses. Almost every analysis reads that cohort through its final answers. But long before the answers exist, the cohort already has an internal organization — some rollouts compute alike, others take genuinely different routes. In a sparse Mixture-of-Experts model that organization is visible for free, in the routing traces that arrive with every token.

Three panels, one per architecture (OSS-20B, OSS-120B, Qwen3-30B-A3B), each plotting median routing effective rank against token budget on a log axis, with one curve per reasoning-effort tier. Every curve rises from a low value at small budgets to a maximum at intermediate budgets and falls again by the largest budget; the maximum moves later as reasoning effort increases. — Fig. 1The shape, measured. One panel per architecture, one curve per reasoning-effort tier (medians). Routing effective rank rises from a concentrated start, peaks at an intermediate budget, and reconcentrates by the end — and the peak arrives *later* with more effort in all four architectures. Compare timing and shape *within* a panel: absolute levels depend on the routing representation and cohort size.

01

The cohort is the unit

Self-consistency aggregates it and group-based training treats it as one object — yet it is almost always summarized by votes and margins, never by its own internal geometry.

02

Routing is a free probe

Expert routing is a by-product of decoding, not an added instrument. It arrives with every token, can be read online at any prefix, and needs no labels, no execution, and no answer text.

03

Read the spectrum

Build a similarity graph across same-question rollouts and its effective rank says how many independent routing directions the cohort really occupies — a single number that turns out to move on a schedule.

How it is measured

Compare who fires together. Take the spectrum. Read its effective rank.

Every step is activation-only. Answer annotations never enter the similarity — they are used solely to locate the commit window and to label same-answer pairs afterwards, so the diagnostic can be computed during serving, before anything is graded.

Three panels. (1) Build the cohort graph: a rollout-by-rollout heatmap of weighted-Jaccard similarity between expert-routing histograms. (2) Read the phase: the effective rank plotted against token budget rises from a low value through a labelled peak and falls again, with the rising side marked divergence and the falling side consolidation. (3) Allocate landmarks: a single question's effective-rank curve with a running maximum and a gate that fires when the curve drops twenty percent below it. — Fig. 2From traces to a phase reading. **(1)** Each pair of rollouts is compared by weighted-Jaccard overlap of their expert-routing histograms, giving the cohort graph W. **(2)** The entropy of W's spectrum gives d_eff, whose arc separates a *divergence* phase from a *consolidation* phase. **(3)** Read along the budget axis, those phases become landmarks a serving system could act on.

W_ij

Compare rollouts by their routing

Sample N = 64 rollouts per question and evaluate on a log budget axis (32 → 32,768 tokens). At each budget, every rollout carries a histogram of router gate weight over (layer, expert) slots; the cohort graph holds their weighted-Jaccard overlap.

W_ij = ∑_e min(h_i^(e), h_j^(e))∑_e max(h_i^(e), h_j^(e))
d_eff

Take the entropy of the spectrum

With p_i the normalized eigenvalues of W, the effective rank is the exponential of the spectral entropy. Unlike ordinary rank it discounts numerically tiny directions; unlike graph density it summarizes the whole eigenvalue distribution. Low means a few shared patterns explain the graph; high means more independent directions are needed.

d_eff = exp(−∑_i p_i log p_i)
h(m)

Split it into two exact channels

Effective rank can move because the leading eigenmode gains or loses mass, or because the residual spectrum expands. The grouping property of Shannon entropy separates them exactly: with m the leading mode's mass and d_⊥ the residual dimensionality, every segment of the trajectory is attributed to common mode, residual, or their mixing.

log d_eff = h(m) + (1−m) log d_⊥

The arc is a property of aligned routing organization, not of activation volume: permuting expert identities within each rollout — preserving sparsity, per-layer load, and every marginal activation statistic exactly — collapses the prominent-peak rate from 98.5% to 0.0% across all ten configurations. What the spectrum measures is which experts fire together across rollouts.

What the spectrum shows

The arc is reproducible — and it has an anatomy.

Two results carry the paper: the trajectory itself, which appears in almost every individual cohort rather than only in the average, and an exact decomposition that says which part of the spectrum is moving.

①

A reproducible trajectory

Pooled, d_eff runs 5.21 → 18.68 (at 512 tokens) → 5.84. A prominent interior maximum appears in 98.5% of 3,105 cohorts — 97.9–99.3% on every benchmark, ≥92.9% on every configuration, and in every difficulty stratum. Timing is architecture-dependent (median peak 768 / 1,024 / 4,096 tokens by family), yet the ordering across datasets is near-fixed (Kendall W = 0.946).

②

One common mode carries it

The leading eigenvector is almost exactly the cohort-wide agreement direction (⟨v₁, u⟩² median 0.98; it tracks plain graph density at ρ = 0.987). Mass draining out of that mode and returning accounts for 0.67 of the arc's motion; residual dimensionality contributes 0.24, and mixing 0.09.

→

Two coordinates, not one scalar

The residual spectrum is not a restatement of the common mode: a grouped cross-validated fit of log d_⊥ on m explains only 37% of out-of-sample variance, leaving 63% as a second, independent axis of cohort routing geometry.

Three panels. (A) Aligned per-family medians showing that as effective rank rises the common-mode mass falls and the residual dimensionality rises, then both reverse. (B) A dot-and-interval chart of the share of the motion of log d_eff attributed to the common mode, the residual, and mixing, over the full path and over the early and late halves; the common-mode share is largest everywhere and largest of all in the late half. (C) A hexbin of log residual dimensionality against common-mode mass, with a grouped cross-validated R-squared of 0.37. — Fig. 3The anatomy of the arc. **(A)** As d_eff rises, spectral mass drains out of the common mode; as it falls, mass returns. **(B)** The common-mode term carries 0.67 of the motion of log d_eff, the residual 0.24, mixing 0.09 — and it dominates in both halves of the budget grid. **(C)** Residual dimensionality nonetheless keeps 63% of its variance beyond the common mode.

What the two channels do

Effort moves the peak. The common mode carries the meaning.

Once the arc has an anatomy, the two channels can be given jobs. Pre-registered, Bonferroni-corrected tests locate answer-alignment in the common mode, and locate the effect of reasoning effort in when and for how long the cohort stays differentiated — not in how high it peaks.

98.5%of 3,105 cohorts show a prominent interior peak

0.67of the arc carried by one common mode

+2.59octaves the peak is delayed by higher effort

Ten MoE configurations × five math/science benchmarks, 64 rollouts per question. The trajectory survives a frozen transfer to held-out code generation (97.7% prominent-peak rate on 1,582 LiveCodeBench-v5 problems) and a binary expert-set graph — so it can be computed from sparse activation indicators alone, without router gate magnitudes.

Common mode → same-answer recoverability

Across 28,622 adjacent-budget steps (clustered on 318 questions × 10 configurations), a rise in common-mode mass strongly predicts that routing similarity alone recovers which rollouts share a final answer: β = +0.522 [+0.430, +0.609], against a residual partial of just −0.047. A within-question time shuffle collapses both toward zero — the association is temporal, not an artifact of scale.

Effort dilates the regime — it doesn't raise it

Pairing effort tiers within each architecture, higher effort delays the maximum by +2.59 octaves, widens the high-rank interval by +1.29 octaves, and raises high-rank occupancy by +0.102 — the same sign in all four architectures. The registered peak-height contrast spans zero (+0.104 [−0.079, +0.232]), locating the effect in timing and duration.

Four dot-and-interval panels — peak delay in octaves, high-rank occupancy, high-rank width in octaves, and integrated common-mode mass — each with one row per architecture plus a pooled row. Peak delay, occupancy, and width are consistently positive across all four architectures; integrated common-mode mass is consistently negative. — Fig. 4Four readouts of the effort contrast, one row per architecture plus the pooled estimate. Higher effort **delays** the maximum, **widens** the high-rank interval, **raises** occupancy near the peak, and **lowers** integrated common-mode mass — directionally consistent in all four architectures.

A structural monitor, not an answer selector

Routing effective rank describes cohort geometry, not correctness: for picking the answer, answer-side statistics remain stronger. Its value is that it is label-free and available mid-generation. And because the absolute level of d_eff depends on the routing representation, expert count, and cohort size, claims are about shape and timing within an architecture — not levels compared across them.

Measured over 10 MoE configurations · 5 benchmarks · 3,105 cohorts gpt-oss-20B/120B at low·medium·high effort, Qwen3-30B-A3B and Qwen3-Next-80B-A3B (Instruct / Thinking) · AIME'24, AIME'25, BRUMO'25, HMMT'25, GPQA · 64 rollouts per question over a 16-point log budget axis

Watch

Concentration, differentiation, and back — a visual essay

A silent vector essay, drawn live in your browser — no video file. A cohort fans out, its routing graph is built from expert overlap alone, and the spectrum's effective rank rises and falls: concentrated, differentiated, concentrated again.

Cite

Reference

From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts

Fudan University author list released after anonymous review

Correspondence: kchen24@m.fudan.edu.cn

Links: Paper (PDF) · Code · arXiv

@misc{deff2026routingrank,
  title   = {From Concentration to Differentiation and Back:
             Routing Effective Rank in MoE Reasoning Cohorts},
  author  = {Anonymous},
  year    = {2026},
  note    = {Preprint; under anonymous review},
  url     = {https://CckFdu.com/deff},
}

Concentration → Differentiation → and back.