arXiv:2505.09246cs.IRcs.AI2025-05被引 2

用四步聚焦检索,让多跳问答更准更快

Autofocus Retrieval: An Effective Pipeline for Multi-Hop Question Answering With Semi-Structured Knowledge

  • 分四步聚焦检索,结合结构与文本信息
  • 首答命中率比第二名高32.1%,跨三基准领先
  • 适合需要精准多跳推理的系统开发者

在真实场景中,机器学习模型常同时接触结构化知识(如知识图谱)和非结构化文本。半结构化知识库(SKBs)将二者关联起来。本文提出AF-Retriever,一种基于SKB的多跳问答框架,通过创新的集成步骤与优化,在所有三个STaRK QA基准上实现零样本和单样本最佳性能。其平均首答命中率超越第二名32.1%。核心机制包括:(1) 使用可替换的大语言模型提取实体属性与关系约束,用于解析与重排序前k个答案;(2) 采用向量相似性搜索对提取的实体与最终答案进行排序;(3) 提出一种新型增量作用域扩展流程,筛选满足约束条件的最佳候选集;(4) 混合检索策略降低错误传播风险。整体流程如同自动对焦,经过四步约束检索后,再经四步处理补充与排序。消融实验与误差分析揭示各组件贡献,包括三种不同LLM重排序策略的对比。代码已开源:https://github.com/kramerlab/AF-Retriever。

原文摘要 · Abstract (English)

In many real-world settings, machine learning models and interactive systems have access to both structured knowledge, e.g., knowledge graphs or tables, and unstructured content, e.g., natural language documents. Yet, most rely on either. Semi-Structured Knowledge Bases (SKBs) bridge this gap by linking unstructured content to nodes within structured data. In this work, we present Autofocus-Retriever (AF-Retriever), a modular framework for SKB-based, multi-hop question answering. It combines structural and textual retrieval through novel integration steps and optimizations, achieving the best zero- and one-shot results across all three STaRK QA benchmarks, which span diverse domains and evaluation metrics. AF-Retriever's average first-hit rate surpasses the second-best method by 32.1%. Its performance is driven by (1) leveraging exchangeable large language models (LLMs) to extract entity attributes and relational constraints for both parsing and reranking the top-k answers, (2) vector similarity search for ranking both extracted entities and final answers, (3) a novel incremental scope expansion procedure that prepares for the reranking on a configurable amount of suitable candidates that fulfill the given constraints the most, and (4) a hybrid retrieval strategy that reduces error susceptibility. In summary, while constantly adjusting the focus like an optical autofocus, AF-Retriever delivers a configurable amount of answer candidates in four constraint-driven retrieval steps, which are then supplemented and ranked through four additional processing steps. An ablation study and a detailed error analysis, including a comparison of three different LLM reranking strategies, provide component-level insights. The source code is available at https://github.com/kramerlab/AF-Retriever .

多跳问答知识库检索增强LLM应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。