将维基百科多实体问答信息结构化，提升大模型推理准确率
SRAG: Structured Retrieval-Augmented Generation for Multi-Entity Question Answering over Wikipedia Graph
- 把检索到的实体整理成带字段的表格，便于模型分析
- 在维基百科多实体问答任务上准确率提升29.6%
- 适合需要整合多源信息的复杂问答场景
多实体问答（MEQA）对大语言模型（LLMs）构成重大挑战，因其难以整合跨多个文档的分散信息。例如，“IEEE会士在不同研究领域中的分布情况？”这类问题需从多个维基百科页面中获取信息。现有检索增强生成（RAG）方法受限于LLM聚合大量页面信息的能力。为此，本文提出结构化检索增强生成（SRAG）框架，将提取的实体系统性地组织为关系型表格（如包含“姓名”“研究领域”等字段），并采用基于表格的推理技术。该方法分离了检索与推理过程，使LLM能专注于结构化数据处理而非原始文本整合。在基于维基百科的多实体问答任务上，实验表明SRAG显著优于现有长上下文LLM和RAG方案，准确率提升29.6%。结果验证了将非结构化数据结构化可有效增强LLM的推理能力。
原文摘要 · Abstract (English)
Multi-entity question answering (MEQA) poses significant challenges for large language models (LLMs), which often struggle to consolidate scattered information across multiple documents. An example question might be "What is the distribution of IEEE Fellows among various fields of study?", which requires retrieving information from diverse sources e.g., Wikipedia pages. The effectiveness of current retrieval-augmented generation (RAG) methods is limited by the LLMs' capacity to aggregate insights from numerous pages. To address this gap, this paper introduces a structured RAG (SRAG) framework that systematically organizes extracted entities into relational tables (e.g., tabulating entities with schema columns like "name" and "field of study") and then apply table-based reasoning techniques. Our approach decouples retrieval and reasoning, enabling LLMs to focus on structured data analysis rather than raw text aggregation. Extensive experiments on Wikipedia-based multi-entity QA tasks demonstrate that SRAG significantly outperforms state-of-the-art long-context LLMs and RAG solutions, achieving a 29.6% improvement in accuracy. The results underscore the efficacy of structuring unstructured data to enhance LLMs' reasoning capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。