用可信度机制让大模型识别并拒绝不确定回答，减少幻觉。
Credal Transformer: A Principled Approach for Quantifying and Mitigating Hallucinations in Large Language Models
- 将注意力机制改为可信度集合，动态反映模型不确定性
- 在无法回答的问题上显著降低错误自信，提升拒绝率
- 适合对可靠性要求高的问答与医疗等关键场景
大型语言模型常产生看似正确实则错误的断言。我们认为这源于Transformer的Softmax函数将模糊注意力分数压缩为单一概率分布，造成‘人为确定性’，丢失了不确定性信息。为此，我们提出可信度Transformer，用基于证据理论的可信度注意力机制（CAM）替代标准注意力。CAM生成一个‘可信度集’（一组分布），其大小直接衡量模型不确定性。我们将注意力分数重新理解为狄利克雷分布的证据质量：证据充足时恢复标准注意力，证据不足时生成扩散分布，体现模糊性。实验表明，该模型能有效识别分布外输入，量化不确定性，并在无法回答的问题上通过拒答显著减少高自信错误。本工作提出一种新架构和可融入模型的不确定性量化设计范式，为更可靠的AI奠定基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) hallucinate, generating factually incorrect yet confident assertions. We argue this stems from the Transformer's Softmax function, which creates "Artificial Certainty" by collapsing ambiguous attention scores into a single probability distribution, discarding uncertainty information at each layer. To fix this, we introduce the Credal Transformer, which replaces standard attention with a Credal Attention Mechanism (CAM) based on evidential theory. CAM produces a "credal set" (a set of distributions) instead of a single attention vector, with the set's size directly measuring model uncertainty. We implement this by re-conceptualizing attention scores as evidence masses for a Dirichlet distribution: sufficient evidence recovers standard attention, while insufficient evidence yields a diffuse distribution, representing ambiguity. Empirically, the Credal Transformer identifies out-of-distribution inputs, quantifies ambiguity, and significantly reduces confident errors on unanswerable questions by abstaining. Our contribution is a new architecture to mitigate hallucinations and a design paradigm that integrates uncertainty quantification directly into the model, providing a foundation for more reliable AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。