用语义不确定性引导文档邻域搜索，提升复杂问答召回率
SUNAR: Semantic Uncertainty based Neighborhood Aware Retrieval for Complex QA
- 基于LLM生成答案候选的不确定性，动态调整文档优先级
- 在两个数据集上相比现有方法最高提升31.84%准确率
- 适合需要高召回率的复杂问答系统研究者使用
复杂问答系统在处理多维度问题时面临显著挑战，尤其在第一阶段检索中难以获取全部相关文档，导致后续推理性能下降。尽管大语言模型（LLMs）提升了推理能力，但受限于上下文窗口，仍存在召回率不足的问题。本文提出SUNAR，一种基于语义不确定性的邻域感知检索方法。该方法利用LLM生成的答案候选，动态评估文档的不确定性，并在文档邻域图中迭代优化检索路径。实验表明，SUNAR在两个复杂QA数据集上显著优于现有基准方法，在最佳情况下性能提升达31.84%。
原文摘要 · Abstract (English)
Complex question-answering (QA) systems face significant challenges in retrieving and reasoning over information that addresses multi-faceted queries. While large language models (LLMs) have advanced the reasoning capabilities of these systems, the bounded-recall problem persists, where procuring all relevant documents in first-stage retrieval remains a challenge. Missing pertinent documents at this stage leads to performance degradation that cannot be remedied in later stages, especially given the limited context windows of LLMs which necessitate high recall at smaller retrieval depths. In this paper, we introduce SUNAR, a novel approach that leverages LLMs to guide a Neighborhood Aware Retrieval process. SUNAR iteratively explores a neighborhood graph of documents, dynamically promoting or penalizing documents based on uncertainty estimates from interim LLM-generated answer candidates. We validate our approach through extensive experiments on two complex QA datasets. Our results show that SUNAR significantly outperforms existing retrieve-and-reason baselines, achieving up to a 31.84% improvement in performance over existing state-of-the-art methods for complex QA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。