arXiv:2603.22651cs.AIcs.CL2026-03被引 6

对比四种AI架构处理财报,找出成本与准确率的最佳平衡点。

Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies

  • 设计四种多智能体协作模式,系统测试其在财报中的表现。
  • 自修正架构准确率最高(F1 0.943),但成本是基线的2.3倍。
  • 分层架构在成本与准确率间最均衡,适合金融领域实际部署。

大语言模型在财务文档结构化信息提取中的应用快速普及,但生产部署面临架构选择难题且缺乏实证指导。本文系统比较了四种多智能体编排架构:串行流水线、并行分发合并、分层主从和反射式自我修正循环。在包含10,000份美股财报(10-K、10-Q、8-K)的语料库上,对五款前沿及开源权重的LLM进行评估,覆盖25个抽取字段,涵盖治理结构、高管薪酬和财务指标,从五个维度衡量:字段级F1、文档级准确率、端到端延迟、每文档成本和每令牌效率。结果表明,反射式架构获得最高字段级F1(0.943),但成本为串行基线的2.3倍;分层架构位于成本-准确率帕累托前沿最优位置(F1 0.921,成本1.4倍)。通过消融实验验证语义缓存、模型路由与自适应重试策略,发现混合配置可在仅1.15倍基线成本下实现反射架构89%的准确率提升。从每天1,000至100,000文档的扩展分析揭示非直观的吞吐量-准确率退化曲线,为受监管金融环境中的多智能体系统部署提供可操作建议。

原文摘要 · Abstract (English)

The adoption of large language models (LLMs) for structured information extraction from financial documents has accelerated rapidly, yet production deployments face fundamental architectural decisions with limited empirical guidance. We present a systematic benchmark comparing four multi-agent orchestration architectures: sequential pipeline, parallel fan-out with merge, hierarchical supervisor-worker and reflexive self-correcting loop. These are evaluated across five frontier and open-weight LLMs on a corpus of 10,000 SEC filings (10-K, 10-Q and 8-K forms). Our evaluation spans 25 extraction field types covering governance structures, executive compensation and financial metrics, measured along five axes: field-level F1, document-level accuracy, end-to-end latency, cost per document and token efficiency. We find that reflexive architectures achieve the highest field-level F1 (0.943) but at 2.3x the cost of sequential baselines, while hierarchical architectures occupy the most favorable position on the cost-accuracy Pareto frontier (F1 0.921 at 1.4x cost). We further present ablation studies on semantic caching, model routing and adaptive retry strategies, demonstrating that hybrid configurations can recover 89\% of the reflexive architecture's accuracy gains at only 1.15x baseline cost. Our scaling analysis from 1K to 100K documents per day reveals non-obvious throughput-accuracy degradation curves that inform capacity planning. These findings provide actionable guidance for practitioners deploying multi-agent LLM systems in regulated financial environments.

多智能体金融文本成本优化架构评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。