让大模型学会在压缩上下文后无法回答时主动拒绝，避免胡说。
Compression-Aware Abstention: Teaching LLMs to Refuse When KV-Compression Masks Remove Answer Evidence
- 训练模型根据关键证据是否保留，决定是否回答
- 压缩下幻觉减少97%，保留证据时答对率不变
- 效果来自证据内容而非输入长度，适合长文本推理
KV缓存压缩通过剔除上下文标记来降低大模型推理内存开销，但当被剔除的标记包含答案依据时，模型可能产生幻觉而非意识到上下文不足。本文首次将压缩感知拒答建模为学习问题：模型学习在支持性证据保留时回答，在证据被压缩移除时拒答。我们基于压缩器存活掩码与精确答案段，标注数据为“自信”（证据保留）或“拒答”（证据移除）。一个10.1M参数的LoRA适配器在约2.6K个MuSiQue 2跳问答样本上训练，可在提示式截断下使基线模型幻觉降低97%，同时保持在证据保留例题上的正确回答能力。相比仅靠提示的拒答基线（过早拒答大量可答例题），该适配器学习到条件化策略。在真实压缩缓存解码中，多压缩器联合训练使基线模型在保留证据例题上获得6-22倍相对提升。受控删除实验表明，模型行为由证据内容驱动，而非输入长度本身。
原文摘要 · Abstract (English)
KV-cache compression reduces LLM inference memory by evicting context tokens, but when the evicted tokens contain answer-bearing evidence, the model may hallucinate instead of recognizing that the compressed context is insufficient. We address this failure from a behavioral perspective: to our knowledge, this is the first work to formulate compression-aware abstention as a learning problem, in which a model learns to answer when supporting evidence survives compression and abstain when it does not. We construct supervision from compressor survival masks and tight answer-bearing spans, labeling examples as Confident when evidence survives and Abstain when it is removed. A 10.1M-parameter LoRA adapter trained on ~2.6K MuSiQue 2-hop QA examples reduces base-model hallucinations by 97% under prompt-style truncation while preserving correct answering on evidence-retaining examples. Unlike prompt-only abstention baselines, which over-abstain on many answerable high-retention examples, the trained adapter learns a conditional policy. We also evaluate the method under actual compressed-cache decoding, where multi-compressor training yields a 6-22x relative lift over the unaided base on evidence-retaining examples. Controlled-deletion experiments show that the learned behavior is driven by evidence content rather than input length alone.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。