发现提示词缓存冗余源于格式而非内容，可用中性模板替代
Probing the Prompt KV Cache: Where It Becomes Dispensable
- 通过分层截断实验定位冗余发生位置和时机
- 用中性填充模板替换高层提示缓存，精度几乎无损
- 适用于Qwen3、Gemma3、Llama3等主流模型家族
现有KV缓存压缩方法表明解码过程中提示词缓存部分冗余，可丢弃或摘要而保持精度。本文探究该冗余何时何地出现：在哪些层级、解码多少步后、以何种形式可被替代而不影响任务。通过控制性剪切干预，发现冗余本质是形式（对话模板结构）而非内容。将高层提示缓存替换为使用中性填充内容的模板缓存，可恢复接近原始精度；而清空相同位置则导致精度崩溃。该现象在Qwen3、Gemma3和Llama3系列模型上，多个数据集均复现。
原文摘要 · Abstract (English)
Prior KV cache compression schemes empirically demonstrate that the prompt cache is partially redundant during decoding, dropping or summarising entries with little accuracy loss. We ask when and what kind of redundancy: at which layers, after how many decoding steps, and in what form can the prompt span KV cache be replaced without breaking the task. A controlled splice intervention swept over layer cutoff and decoding steps shows this redundancy is about form (chat template scaffolding) rather than content. Replacing the upper layer prompt span KV cache with KV cache from a chat template scaffold whose user content is a neutral filler recovers near clean accuracy, while zeroing the same slots collapses accuracy. The dissociation replicates across the Qwen3, Gemma 3, and Llama 3 families on multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。