提出记忆解码流水线，3倍提速且不丢检测精度。
Decoding Memories: An Efficient Pipeline for Self-Consistency Hallucination Detection
- 发现自一致性生成中前缀令牌冗余，仅需部分重算
- 实现最高3倍加速，AUROC指标不变
- 适合需要高效多响应生成的场景
大型语言模型在科研与实际应用中表现优异，但仍存在幻觉问题。现有检测方法在句子级生成上表现不佳，或依赖领域知识。自一致性方法虽有改进，但因重复生成导致计算开销高。本文首次研究自一致性方法中的冗余现象，发现生成过程中共享前缀令牌，非精确答案部分对语义贡献极小。基于此，提出新型解码记忆流水线（DMP），通过选择性推理与退火解码提升效率。该方法与模型、数据集、解码策略及自一致性基线无关，可稳定提升多响应生成效率，未来有望扩展至对齐与推理任务。大量实验表明，本方法在不牺牲AUROC性能的前提下，实现最高3倍加速。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated impressive performance in both research and real-world applications, but they still struggle with hallucination. Existing hallucination detection methods often perform poorly on sentence-level generation or rely heavily on domain-specific knowledge. While self-consistency approaches help address these limitations, they incur high computational costs due to repeated generation. In this paper, we conduct the first study on identifying redundancy in self-consistency methods, manifested as shared prefix tokens across generations, and observe that non-exact-answer tokens contribute minimally to the semantic content. Based on these insights, we propose a novel Decoding Memory Pipeline (DMP) that accelerates generation through selective inference and annealed decoding. Being orthogonal to the model, dataset, decoding strategy, and self-consistency baseline, our DMP consistently improves the efficiency of multi-response generation and holds promise for extension to alignment and reasoning tasks. Extensive experiments show that our method achieves up to a 3x speedup without sacrificing AUROC performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。