让大模型生成时精准标注每句话来源,提升科学场景可信度。
SAFE: Improving LLM Systems using Sentence-Level In-generation Attribution
- 生成时逐句预测并标注引用来源,支持即时验证
- 在干净数据集上使各类算法准确率提升2.1%至6.0%
- 适用于千句级真实文档,具备强泛化能力
大型语言模型(LLMs)在多个科学领域应用日益广泛,但其广泛应用仍受限于输出缺乏可信赖、可验证的来源归属。当前模型常无法提供可靠引用,甚至出现错误归因,在科学与高风险场景中构成障碍。为实现可信输出,归属系统需在短文本层面具备高精度,即对文档中的具体句子进行定位,而非整篇文档。本文提出SAFE框架,用于检索增强生成(RAG)系统中生成过程中的句子级归因。该框架包含两个步骤:预测句子所需参考文献数量,以及完成句子归因。方法在第一步达到95%准确率,并使所有归因算法在清洁数据集上的准确率(按最大可能准确率归一化)提升2.1%~6.0%,优于其顶1准确率。在包含数百至数千句子的真实文档中应用后,SAFE仍能可靠地将句子归因到原始文档,证明其超越基准测试的泛化能力。SAFE框架及训练数据已公开于GitHub。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly applied in various science domains, yet their broader adoption remains constrained by a critical challenge: the lack of trustworthy, verifiable outputs. Current LLMs often generate answers without reliable source attribution, or worse, with incorrect attributions, posing a barrier to their use in scientific and high-stakes settings, where traceability and accountability are paramount. To be reliable, attribution systems require high accuracy for short-length attribution on retrieved data, i.e., attribution to a sentence within a document rather than the entire document. We propose SAFE, a Sentence-level A ttribution FramEwork for Retrieve-Augmented Generation (RAG) systems that attributes generated sentences during generation. This allows users to verify sentences as they read them and correct the model when the attribution indicates the generated text is not grounded in the documents, increasing the safety of LLM systems. This framework consists of two steps: predicting the required number of references for a sentence, and attributing the sentence. Our approach achieved 95% accuracy in the first step, which translated to 2.1\~6.0% improvements in the accuracy (normalized for maximum possible accuracy) of all attribution algorithms in our clean dataset, when compared to their top-1 accuracy. We also applied SAFE in real-world scenarios with documents containing hundreds to thousands of sentences. In these settings, SAFE reliably attributed sentences to their source documents, demonstrating that the method generalizes beyond controlled benchmarks. The SAFE framework and the training dataset are publicly available on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。