量化检索缓存会损害生成内容的忠实性，即使答案正确也可能失真。
Faithfulness Is Not Free: Auditing Offline KV-Cache Quantization in Retrieval-Augmented Generation
- 用自然语言推断和大模型判断评估压缩后缓存的忠实性
- INT4量化使90%以上正确答案的可信度下降，准确性指标无法发现
- 在噪声检索或更多文档片段时，损害更严重，需提前审计
检索增强生成系统可通过预计算并存储检索文档的键值缓存，避免每次查询重新编码上下文。对这些缓存进行量化可进一步减少存储，但现有工作未关注压缩是否损害生成结果的忠实性——即回答是否仍基于原始检索证据。忠实性与准确性不等价：模型可能给出正确答案，但已不再受输入上下文支持。本文在RGB和HotpotQA数据集上，对Qwen2.5-7B-Instruct进行INT8和INT4量化，使用幻觉检测器、NLI蕴含关系及大模型评判工具，同时评估准确性和忠实性。结果显示，INT8量化在两项指标上接近无损；而INT4量化虽仅轻微降低准确性，却导致超过90%的正确答案出现忠实性下降，即准确率指标无法察觉此类退化。该问题在噪声检索和更多检索段落场景下加剧。因此，部署压缩缓存前必须审计忠实性。
原文摘要 · Abstract (English)
Retrieval-augmented generation systems can precompute and store key-value caches of retrieved documents to avoid re-encoding context at every query. Quantizing these caches further reduces storage, but no prior work asks whether compression damages faithfulness, whether responses remain grounded in the retrieved evidence. Faithfulness and accuracy are not equivalent: a model can produce a correct answer that is no longer supported by the context it was given. We evaluate Qwen2.5-7B-Instruct under INT8 and INT4 quantization on RGB and HotpotQA, measuring both accuracy and faithfulness with a hallucination detector, NLI entailment, and an LLM judge. INT8 is near-lossless across both metrics. INT4 reduces accuracy and, more critically, even among answers that remain factually correct, over 90% of faithfulness changes are negative, i.e., accuracy metrics are blind to this regression. The harm grows under noisy retrieval and with more retrieved chunks. Faithfulness must be audited before compressed caches are deployed.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。