让大模型学会在没把握时主动放弃回答，提升可信度。
When to Speak, When to Abstain: Contrastive Decoding with Abstention
- 通过对比分析上下文与参数知识相关性，决定是否生成答案
- 在无信息时正确放弃回答，避免胡编乱造
- 无需训练即可增强模型可靠性，适合高风险场景
大型语言模型（LLMs）通过利用预训练参数知识和外部上下文知识，在多种任务中表现优异。尽管对两种知识的利用已有大量研究，但在缺乏相关信息时的表现仍被忽视。为此，我们构建了一个包含四种知识获取情境的受控测试环境，发现传统使用方式在所有情况下均缺乏鲁棒性。针对此问题，我们提出无需训练的对比解码与拒答方法（CDA），使模型在有相关知识时生成回答，否则选择拒答。CDA评估输入与两种知识源的相关性，自适应决定优先使用哪种信息并排除另一种。大量实验表明，CDA能同时实现精准生成与合理拒答，显著提升可靠性并维护用户信任。
原文摘要 · Abstract (English)
Large Language Models (LLMs) demonstrate exceptional performance across diverse tasks by leveraging pre-trained (i.e., parametric) and external (i.e., contextual) knowledge. While substantial efforts have been made to enhance the utilization of both forms of knowledge, situations in which models lack relevant information remain underexplored. To investigate this challenge, we first present a controlled testbed featuring four distinct knowledge access scenarios, including the aforementioned edge case, revealing that conventional LLM usage exhibits insufficient robustness in handling all instances. Addressing this limitation, we propose Contrastive Decoding with Abstention (CDA), a novel training-free decoding method that allows LLMs to generate responses when relevant knowledge is available and to abstain otherwise. CDA estimates the relevance of both knowledge sources for a given input, adaptively deciding which type of information to prioritize and which to exclude. Through extensive experiments, we demonstrate that CDA can effectively perform accurate generation and abstention simultaneously, enhancing reliability and preserving user trust.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。