让大模型先思考再回答,减少幻觉错误。
Hallucinate Less by Thinking More: Aspect-Based Causal Abstention for Large Language Models
- 通过因果推断分析模型内部知识多样性,提前判断是否该跳过回答。
- 在多个基准测试中,准确率提升显著,优于现有方法。
- 适合需要高可靠性的问答系统,如医疗、法律等场景。
大型语言模型(LLMs)常生成流畅但事实错误的回答,即幻觉现象。回避机制(abstention)通过输出“我不知道”等方式防范错误,但现有方法多依赖生成后的信号(如生成差异或反馈),难以提前预防不可靠回答。本文提出基于方面因果回避(Aspect-Based Causal Abstention, ABCA)框架,通过因果推断分析模型内部知识的多面性,反映其从不同来源习得的知识维度(如学科、法律语境、时间范围)。ABCA基于这些方面估计因果效应,评估查询相关知识的可靠性,支持两类早期回避:类型1为各方面效应不一致(知识冲突),类型2为各方面一致支持回避(知识不足)。在标准基准上的实验表明,ABCA提升了回避可靠性,达到当前最优性能,并增强了决策可解释性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often produce fluent but factually incorrect responses, a phenomenon known as hallucination. Abstention, where the model chooses not to answer and instead outputs phrases such as "I don't know", is a common safeguard. However, existing abstention methods typically rely on post-generation signals, such as generation variations or feedback, which limits their ability to prevent unreliable responses in advance. In this paper, we introduce Aspect-Based Causal Abstention (ABCA), a new framework that enables early abstention by analysing the internal diversity of LLM knowledge through causal inference. This diversity reflects the multifaceted nature of parametric knowledge acquired from various sources, representing diverse aspects such as disciplines, legal contexts, or temporal frames. ABCA estimates causal effects conditioned on these aspects to assess the reliability of knowledge relevant to a given query. Based on these estimates, we enable two types of abstention: Type-1, where aspect effects are inconsistent (knowledge conflict), and Type-2, where aspect effects consistently support abstention (knowledge insufficiency). Experiments on standard benchmarks demonstrate that ABCA improves abstention reliability, achieves state-of-the-art performance, and enhances the interpretability of abstention decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。