让AI引用更精准,只标关键信息片段
Concise and Sufficient Sub-Sentence Citations for Retrieval-Augmented Generation
- 提出细粒度子句级引用,精准定位答案来源
- 构建专用数据集并用LLM自动标注训练数据
- 生成的引用更简洁可读,便于用户验证真伪
在检索增强生成(RAG)问答系统中,为大语言模型输出添加引用可提升可验证性并帮助用户识别幻觉。然而现有归因方法存在两个问题:一是引用通常以句子或段落为单位,可能包含大量无关内容;二是句子级引用可能遗漏关键验证信息,迫使用户阅读上下文。本文提出生成既简洁又充分的子句级引用,降低用户验证成本。我们首先制定标注规范并构建相应数据集,进而提出一个符合标准的归因框架。该框架利用大模型自动生成微调数据,并通过信用模型过滤低质量样本。在构建数据集上的实验表明,该方法能生成高质量且更易读的引用。
原文摘要 · Abstract (English)
In retrieval-augmented generation (RAG) question answering systems, generating citations for large language model (LLM) outputs enhances verifiability and helps users identify potential hallucinations. However, we observe two problems in the citations produced by existing attribution methods. First, the citations are typically provided at the sentence or even paragraph level. Long sentences or paragraphs may include a substantial amount of irrelevant content. Second, sentence-level citations may omit information that is essential for verifying the output, forcing users to read the surrounding context. In this paper, we propose generating sub-sentence citations that are both concise and sufficient, thereby reducing the effort required by users to confirm the correctness of the generated output. To this end, we first develop annotation guidelines for such citations and construct a corresponding dataset. Then, we propose an attribution framework for generating citations that adhere to our standards. This framework leverages LLMs to automatically generate fine-tuning data for our task and employs a credit model to filter out low-quality examples. Our experiments on the constructed dataset demonstrate that the propose approach can generate high-quality and more readable citations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。