arXiv:2501.18539cs.CLcs.AI2025-01ACL被引 15

让大模型一次性精准检索多源信息,提升复杂问题回答准确率

Can we Retrieve Everything All at Once? ARM: An Alignment-Oriented LLM-based Retrieval Method

  • 用大模型分析问题与数据结构的关系,实现全局检索
  • 在Bird和OTT-QA上比传统方法最高提升15.9和19.3个百分点
  • 适合需要跨文档、多步推理的开放域问答场景

真实世界的开放域问题往往复杂,需从多个信息源获取答案。虽然大语言模型(LLM)能将复杂任务分解为简单步骤,但其问题分解缺乏对可用数据及组织结构的认知,导致检索效果不佳。现有基于代理的RAG采用迭代式检索,依赖前序结果生成后续查询,探索效率低。为此,本文提出一种基于大模型的检索方法ARM,通过挖掘数据对象间的关联关系,使问题与数据组织结构更好对齐,实现复杂问题的一次性全量检索。在Bird和OTT-QA两个数据集上的实验表明,ARM在执行准确率上比标准RAG最高提升5.2个百分点,比ReAct型agentic RAG提升15.9个百分点;在OTT-QA上F1匹配分数分别高出5.5和19.3个百分点。

原文摘要 · Abstract (English)

Real-world open-domain questions can be complicated, particularly when answering them involves information from multiple information sources. LLMs have demonstrated impressive performance in decomposing complex tasks into simpler steps, and previous work has used it for better retrieval in support of complex questions. However, LLM's decomposition of questions is unaware of what data is available and how data is organized, often leading to a sub-optimal retrieval performance. Recent effort in agentic RAG proposes to perform retrieval in an iterative fashion, where a followup query is derived as an action based on previous rounds of retrieval. While this provides one way of interacting with the data collection, agentic RAG's exploration of data is inefficient because successive queries depend on previous results rather than being guided by the organization of available data in the collection. To address this problem, we propose an LLM-based retrieval method -- ARM, that aims to better align the question with the organization of the data collection by exploring relationships among data objects beyond matching the utterance of the query, thus leading to a retrieve-all-at-once solution for complex queries. We evaluated ARM on two datasets, Bird and OTT-QA. On Bird, it outperforms standard RAG with query decomposition by up to 5.2 pt in execution accuracy and agentic RAG (ReAct) by up to 15.9 pt. On OTT-QA, it achieves up to 5.5 pt and 19.3 pt higher F1 match scores compared to these approaches.

检索增强大模型复杂问答

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。