arXiv:2410.16708cs.CL2024-10被引 21

将答案拆成原子事实,提升问答可信度与精准溯源。

Atomic Fact Decomposition Helps Attributed Question Answering

  • 用指令微调大模型拆解长答案为原子事实
  • 在多个数据集上超越现有方法,证据定位更准
  • 适合需要高可信度与可解释性的问答场景

属性化问答(AQA)旨在为问题提供可信答案及可靠的证据溯源报告。当前主流方法包括检索后阅读(RTR)和事后检索,但前者易引入无关知识,后者难以理解复杂逻辑并精准定位需修改内容。本文提出基于原子事实分解的检索与编辑框架(ARE),通过指令微调的大模型将生成的长文本答案分解为分子语句和原子事实,其训练数据来自大规模知识图谱的单跳邻居提取与自然语言转换。随后,系统利用搜索引擎检索相关证据,并由大模型验证是否需扩展或修改。修改后的事实回溯至原答案,依据分子语句与原子事实的关系聚合证据。大量实验表明,该方法在多个数据集上优于现有最先进模型,并引入新指标 $Attr_{p}$ 评估证据定位精度。

原文摘要 · Abstract (English)

Attributed Question Answering (AQA) aims to provide both a trustworthy answer and a reliable attribution report for a given question. Retrieval is a widely adopted approach, including two general paradigms: Retrieval-Then-Read (RTR) and post-hoc retrieval. Recently, Large Language Models (LLMs) have shown remarkable proficiency, prompting growing interest in AQA among researchers. However, RTR-based AQA often suffers from irrelevant knowledge and rapidly changing information, even when LLMs are adopted, while post-hoc retrieval-based AQA struggles with comprehending long-form answers with complex logic, and precisely identifying the content needing revision and preserving the original intent. To tackle these problems, this paper proposes an Atomic fact decomposition-based Retrieval and Editing (ARE) framework, which decomposes the generated long-form answers into molecular clauses and atomic facts by the instruction-tuned LLMs. Notably, the instruction-tuned LLMs are fine-tuned using a well-constructed dataset, generated from large scale Knowledge Graphs (KGs). This process involves extracting one-hop neighbors from a given set of entities and transforming the result into coherent long-form text. Subsequently, ARE leverages a search engine to retrieve evidences related to atomic facts, inputting these evidences into an LLM-based verifier to determine whether the facts require expansion for re-retrieval or editing. Furthermore, the edited facts are backtracked into the original answer, with evidence aggregated based on the relationship between molecular clauses and atomic facts. Extensive evaluations demonstrate the superior performance of our proposed method over the state-of-the-arts on several datasets, with an additionally proposed new metric $Attr_{p}$ for evaluating the precision of evidence attribution.

问答系统大模型知识图谱可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。