解决大模型生成回答却漏引参考文献的问题,提升可信度。
Citation Failure: Definition, Analysis and Efficient Mitigation
- 分离引用失败与回答错误,聚焦引用缺失问题。
- 发现引用失败随证据关系复杂度上升而加剧,混合方法更有效。
- 提出CITENTION框架,融合生成、注意力和检索提升引用质量。
基于大语言模型的检索增强生成系统本应通过引用简化结果验证,但在引用失败场景下,模型虽能生成有用回答,却无法提供完整证据引用。与以往工作不同,本文将引用失败与回答错误区分开来,前者指回答正确但缺乏引用。为应对该问题,研究分两步:首先分析引用失败发生条件,构建了CITECONTROL基准,系统性地改变回答与证据间的关系,实验表明引用失败随关系复杂度增加而上升,并提示结合多种引用方法可提升性能;其次提出CITENTION框架,整合生成式、注意力机制与检索式引用方法,在CITECONTROL及跨域设置中均显著提升引用效果。数据与代码已公开。
原文摘要 · Abstract (English)
Citations from LLM-based RAG systems are supposed to simplify response verification. However, this goal is undermined in cases of citation failure, where a model generates a helpful response, but fails to generate citations to complete evidence. In contrast to previous work, we propose to disentangle this from response failure, where the response itself is flawed, and citing complete evidence is impossible. To address citation failure, this work follows a two-step approach: (1) We study when citation failure occurs and (2) how it can be mitigated efficiently. For step 1, we extend prior work by investigating how the relation between response and evidence affects citation quality. We introduce CITECONTROL, a benchmark that systematically varies this relation to enable the analysis of failure modes. Experiments show that failures increase with relational complexity and suggest that combining citation methods could improve performance, motivating step 2. To study the efficient improvement of LLM citation, we propose CITENTION, a framework integrating generative, attention-based, and retrieval-based methods. Results demonstrate substantial citation improvements on CITECONTROL and in transfer settings. We make our data and code publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。