通过对比解码减少大模型幻觉，无需重训即可提升回答准确性。
Delta -- Contrastive Decoding Mitigates Text Hallucinations in Large Language Models
- 随机掩码输入并对比输出分布，仅在推理时调整生成策略。
- SQuAD v1.1和v2分别提升3%和6%，TriviaQA等任务提升7%和2%。
- 适合医疗、法律等对准确性要求高的实际应用场景。
大语言模型在自然语言处理中表现强大，但仍易产生幻觉，生成事实错误或虚构内容，影响其在医疗、法律等高风险领域的可靠性。为解决此问题，我们提出Delta，一种无需模型重训练或额外数据的推理阶段方法。Delta通过随机掩码输入提示，并对比原始与掩码输入的输出分布，仅通过推理计算抑制幻觉。我们在包含丰富上下文的问答基准上评估，SQuAD v1.1和v2的准确率分别提升约3%和6%，TriviaQA和Natural Questions在采样解码下分别提升7%和2%。此外，SQuAD v2的无答案精确匹配得分提升超10个百分点，证明其有效缓解由上下文模糊引发的幻觉。结果表明，Delta是一种计算高效且可扩展的提升大模型可靠性的方法。
原文摘要 · Abstract (English)
Large language models (LLMs) demonstrate strong capabilities in natural language processing but remain prone to hallucinations, generating factually incorrect or fabricated content. This issue undermines their reliability, particularly in high-stakes domains such as healthcare and legal advisory. To address this challenge, we propose Delta, an inference-time method that reduces hallucinations without requiring model retraining or additional data. Delta works by randomly masking parts of the input prompt and contrasting the output distributions for the original and masked inputs, effectively suppressing hallucinations through inference-only computations. We evaluate Delta on context-rich question-answering benchmarks, achieving absolute improvements of approximately 3 and 6 percentage points on SQuAD v1.1 and v2, respectively, and 7 and 2 percentage points on TriviaQA and Natural Questions under-sampling decoding. Delta also improves the no-answer exact match score on SQuAD v2 by over ten percentage points, demonstrating its effectiveness in mitigating hallucinations arising from contextual ambiguity. These results highlight Delta as a computationally efficient and scalable approach for improving the reliability of LLMs in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。