金融问答中兼顾检索精度与鲁棒性,提出两阶段混合检索新方法。
Sustainable Hybrid Document-Routed Retrieval for Financial RAG: Resolving the Robustness-Precision Trade-off
- 先用文档路由筛选目标文档,再在内部分块精确检索
- 平均分7.54,错误率6.4%,完美回答率达20.1%
- 兼顾高精度、低失败率和低成本,适合金融场景部署
面向金融文档问答的检索增强生成系统通常采用基于片段的检索方式:将文档切分成片段,嵌入后按相似性检索。但在结构同质的文档集(如监管文件)中,易出现跨文档片段混淆。语义文件路由(SFR)利用大模型结构化输出将查询路由至完整文档,减少灾难性失败,但牺牲了细粒度片段的精准性。我们在FinDER基准(1500个查询,覆盖五个组别)上发现:SFR平均得分6.45,失败率10.3%;而基于片段的检索(CBR)虽失败率22.5%,但能产生13.8%的完美答案。为此,我们提出两阶段混合文档路由检索(HDRR):先以SFR过滤文档,再在选定文档内进行片段级检索,消除跨文档混淆的同时保持高精度。HDRR在所有指标上均最优:平均分7.54(较CBR提升25.2%,较SFR提升16.9%),失败率6.4%,正确率67.7%(较CBR提升18.7个百分点),完美答案率20.1%(较CBR提升6.3个百分点,较SFR提升11.6个百分点),且在五组数据中同时实现最低失败率与最高精度。此外,HDRR更高效:每查询仅需约5K–15K tokens(远低于SFR的50K–200K),无索引时大模型开销(相比一次性的~$100上下文索引成本),且比自校正代理基线少用多次大模型调用,显著降低部署时的API支出与能耗。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) systems for financial document QA typically follow a chunk-based paradigm: documents are split into fragments, embedded, and retrieved by similarity. In structurally homogeneous corpora such as regulatory filings, this suffers from cross-document chunk confusion. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices targeted-chunk precision. We identify this robustness-precision trade-off on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve it, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk retrieval scoped to the identified document(s), eliminating cross-document confusion while preserving chunk precision. HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a 6.4% failure rate, 67.7% correctness (+18.7 pp over CBR), and a 20.1% perfect-answer rate (+6.3 pp over CBR, +11.6 pp over SFR), simultaneously attaining the lowest failure rate and highest precision across all five groups. Beyond accuracy, HDRR is also the most efficient of the high-quality systems: it preserves CBR's compact per-query token budget (~5K-15K, an order of magnitude below SFR's ~50K-200K), incurs no indexing-time LLM spend (versus the one-time ~$100 cost of contextual indexing), and uses fewer per-query LLM calls than self-correcting agentic baselines, translating directly to lower API spend and inference-time energy at deployment scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。