针对需要查遍所有文档的复杂问答,提出结构感知的检索增强生成方法。
PluriHopRAG: Exhaustive, Recall-Sensitive QA Through Corpus-Specific Document Structure Learning
- 基于语料文档结构生成合成样例,分解复杂问题
- 在真实报告数据集上实现40%到52%的准确率提升
- 适合金融、法律等需全面查证的高要求场景
检索增强生成(RAG)常用于单跳或多重跳问答,但在金融、法律、医疗报告等实际场景中,常需穷尽所有文档查找信息且无明确停止条件。本文定义此类问题为‘多跳’(pluri-hop),并提出三个核心要求:召回敏感、完全覆盖与精确性。为此构建了多语言诊断基准PluriHopWIND,包含191篇风电行业真实报告中的48个问题,具有高重复性以模拟真实数据中的干扰项。现有方法在该基准上仅达40%的逐句F1。针对此,提出PluriHopRAG,通过学习语料特异性文档结构生成合成样本,分解查询,并在文档层级使用交叉编码器过滤,减少大模型推理开销。在PluriHopWIND和基于金融、法律、科学报告的Loong基准上验证,对基础大模型的F1提升达18%-52%,相比长上下文推理提升33%,比原始RAG提升52%。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has been used in question answering (QA) systems to improve performance when relevant information is in one (single-hop) or multiple (multi-hop) passages. However, many real life scenarios (e.g. dealing with financial, legal, medical reports) require checking all documents for relevant information without a clear stopping condition. We term these pluri-hop questions, and formalize them by 3 conditions - recall sensitivity, exhaustiveness, and exactness. To study this setting, we introduce PluriHopWIND, a multilingual diagnostic benchmark of 48 pluri-hop questions over 191 real wind-industry reports, with high repetitiveness to reflect the challenge of distractors in real-world datasets. Naive, graph-based, and multimodal RAG methods only reach up to 40% statement-wise F1 on PluriHopWIND. Motivated by this, we propose PluriHopRAG, which learns from synthetic examples to decompose queries according to corpus-specific document structure, and employs a cross-encoder filter at the document level to minimize costly LLM reasoning. We test PluriHopRAG on PluriHopWIND and the Loong benchmark built on financial, legal and scientific reports. On PluriHopWIND, our method shows 18-52% F1 score improvement across base LLMs, while on Loong, we show 33% improvement over long-context reasoning and 52% improvement over naive RAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。