通过自改进机制提升金融问答的准确率与可审计性
Towards Expert Financial QA via Self-Improving RAG
- 分三阶段代理协作，基于反馈动态重试纠错
- 在FinanceBench上达86%准确率，36.4%错误答案被修复
- 每步决策留痕，适合监管合规场景
专家级金融问答需具备事实验证能力以避免数值幻觉，并支持审计追踪以满足监管要求，而标准单次检索增强生成（RAG）系统缺乏这些特性。本文提出Self-Improving RAG框架，将文档问答分解为检索、推理与判别三个专用代理，由编排器协调并实现反馈驱动的自我修正。当判别代理评分低于动态阈值时，系统触发重试：扩大检索范围、优化提示词、放宽接受标准。在FinanceBench（SEC文件问答）上的评估显示，该框架在有金标指导下达到86%准确率，36.4%的初始错误答案通过针对性重试得以恢复，接近四成。关键发现是：固定检索流程配合判别驱动重试即可取得优异效果，且全程可解释。所有决策均记录置信度，满足受监管金融应用的审计需求。
原文摘要 · Abstract (English)
Expert-level financial question answering requires both grounded verification to catch numeric hallucinations and audit trails for regulatory compliance, attributes that standard single-pass RAG systems lack. We take a step toward this goal with Self-Improving RAG, a framework that decomposes document QA into three specialized agents (Retrieval, Reasoning, and Judge) coordinated by an orchestrator with feedback-driven self-correction. When the Judge Agent scores an answer below a dynamic threshold, the system triggers retry with escalated strategies: broader retrieval, more careful prompting, and relaxed acceptance criteria. We evaluate on FinanceBench (SEC filing QA), where Self-Improving RAG achieves 86% oracle-guided accuracy (measuring agreement with gold answers) with a 36.4% Lazarus Rate, recovering nearly 4 in 10 initially incorrect answers through targeted retry. A key finding is that a fixed retrieval pipeline with judge-driven retry achieves strong results without dynamic routing, providing full interpretability. Every decision is logged with confidence scores, enabling the audit trails required for regulated financial applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。