针对金融长文档问答中答错因检索不全的问题,提出分页精检方法提升准确率。
Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

- 将页面作为中间检索单元,用领域微调的双编码器提升分页相关性判断。
- 在150个问题上,页面召回率和段落检索效果显著优于传统方法。
- 适用于对金融文档精准问答有高要求的场景,如合规审查与审计。
检索增强生成在长篇金融监管文件的问答中日益普及,但其可靠性依赖于能否精确获取支持答案的上下文。我们研究了一种常见失败模式:正确文档被检索到,但包含答案的页面或段落未被命中,导致生成模型基于不完整上下文推断。尽管这一在文档内检索失败现象具有重要实践意义,但在金融问答文献中尚未得到系统关注。我们从文档、页面、段落三个粒度评估检索性能,并引入基于最优假设(oracle)的分析以提供检索与生成性能的理论上限。在FinanceBench的150个问题子集上,对比了密集、稀疏、混合及层级检索策略,含重排序与查询重构。结果表明,文档检索提升通常带动页面召回提升,但最优表现仍显示页面与段落级检索存在改进空间。为此,我们提出一个领域微调的页面打分器,将页面作为文档与段落之间的中间检索单元。不同于以往基于段落的层级检索,该方法在金融文件语义连贯性的基础上,专门微调双编码器以实现页面级相关性判别。整体实验表明,该方法显著提升了页面召回率与段落检索精度。
原文摘要 · Abstract (English)
Retrieval-augmented generation is increasingly used for financial question answering over long regulatory filings, yet reliability depends on retrieving the exact context needed to justify answers in high stakes settings. We study a frequent failure mode in which the correct document is retrieved but the page or chunk that contains the answer is missed, leading the generator to extrapolate from incomplete context. Despite its practical significance, this within-document retrieval failure mode has received limited systematic attention in the Financial Question Answering (QA) literature. We evaluate retrieval at multiple levels of granularity, document, page, and chunk level, and introduce an oracle based analysis to provide empirical upper bounds on retrieval and generative performance. On a 150 question subset of FinanceBench, we reproduce and compare diverse retrieval strategies including dense, sparse, hybrid, and hierarchical methods with reranking and query reformulation. Across methods, gains in document discovery tend to translate into stronger page recall, yet oracle performance still suggests headroom for page and chunk level retrieval. To target this gap, we introduce a domain fine-tuned page scorer that treats pages as an intermediate retrieval unit between documents and chunks. Unlike prior passage-based hierarchical retrieval, we fine-tune a bi-encoder specifically for page-level relevance on financial filings, exploiting the semantic coherence of pages. Overall, our results demonstrate a significant improvement in page recall and chunk retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。