arXiv:2606.17034cs.CLcs.LG2026-06被引 4

让大模型高效删除已处理内容,只需改一小段缓存,不重算后续全部内容。

KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing

论文配图:KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing
图 1 · 摘自论文原文
  • 用学习到的引导状态替换待删内容的缓存,其余保持不变
  • 删除后性能接近全重算,在32K上下文上延迟仅增24%
  • 适合需要动态清理长文本中错误信息的场景

在长上下文大模型应用中,事后修改已处理内容极具挑战:一次局部编辑会全局影响缓存状态。若在预填充后发现过时信息、错误工具结果或有害提示,精确删除需重算所有后续 token,开销随后缀长度增长。本文提出 KVEraser,一种可学习的 KV 缓存编辑方法,仅替换被删区间对应的 KV 状态,其余缓存保持不变。通过两阶段训练:通用邻域预训练使模型学会抑制删除段的影响,任务特定微调适配下游场景。实验表明,KVEraser 在 1K–32K 上下文长度的域内任务中表现接近全重算,而延迟仅增加 24%,远低于全重算 17.6 倍的增幅。在含有害事实干扰的未见长文档问答任务中,其性能优于其他近似方法,速度提升 3–4 倍。

原文摘要 · Abstract (English)

Post-hoc context erasing over the KV cache is challenging because a local edit has a global consequence: once a span has been processed, its influence propagates into the cached states of all subsequent tokens. This issue arises naturally in long-context LLM applications, where stale retrieved facts, incorrect tool observations, retracted user preferences, or harmful prompt injections may be identified only after prefill. Exact erasing must then recompute all tokens after the deleted span, making its computational cost depend on suffix length rather than erased-span length. We introduce KVEraser, a learned KV-cache editing method for efficient localized context erasing. Given a processed context and a span to remove, KVEraser replaces only the KV states of the erased interval with learned steering states while reusing the remaining cache unchanged. To learn a transferable erasing mechanism, we build a two-stage training pipeline: generic span-neighbor pre-training teaches the eraser to suppress the influence of the erased span, while task-specific fine-tuning adapts this capability to downstream scenarios. Experiments show that KVEraser nearly matches full recomputation in post-erasure performance on in-domain tasks across 1K--32K context lengths, while its latency increases by only 24% compared with a 17.6x increase for full recomputation. KVEraser also generalizes to unseen long-document QA tasks with harmful factual distractors, achieving the best performance among approximate baselines with a 3--4x speedup over full recomputation.

KV缓存上下文编辑高效推理大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。