提出金融多智能体系统评估框架,强调协作设计比模型大小更重要。
Toward Reliable Evaluation of LLM-Based Financial Multi-Agent Systems: Taxonomy, Coordination Primacy, and Cost Awareness
- 构建四维分类体系,涵盖架构、协作、记忆与工具集成
- 发现协作协议设计影响大于模型规模,可能决定交易成败
- 提出新指标CBS,可检验协作是否真正盈利且抵消交易成本
基于大语言模型的金融交易多智能体系统自2023年以来迅速发展,但该领域缺乏统一的理解框架与可信的评估方法。本文提出三项贡献:第一,构建涵盖架构模式、协作机制、记忆结构和工具集成的四维分类体系,并应用于12个多智能体系统及两个单智能体基线;第二,提出协作主导假说(CPH):智能体间协作协议设计是交易决策质量的核心驱动因素,其影响常超过模型规模;该假说为可证伪的研究假设,依赖现有评估基础设施尚无法验证;第三,指出五类常见评估缺陷(前瞻偏差、幸存者偏差、回测过拟合、忽略交易成本、忽视制度变迁),这些错误可能导致收益符号反转。基于此,提出协调盈亏平衡价差(CBS)指标,用于衡量协作是否在扣除交易成本后仍具真实价值,并建议最低评估标准以支撑对CPH的验证。
原文摘要 · Abstract (English)
Multi-agent systems based on large language models (LLMs) for financial trading have grown rapidly since 2023, yet the field lacks a shared framework for understanding what drives performance or for evaluating claims credibly. This survey makes three contributions. First, we introduce a four-dimensional taxonomy, covering architecture pattern, coordination mechanism, memory architecture, and tool integration; applied to 12 multi-agent systems and two single-agent baselines. Second, we formulate the Coordination Primacy Hypothesis (CPH): inter-agent coordination protocol design is a primary driver of trading decision quality, often exerting greater influence than model scaling. CPH is presented as a falsifiable research hypothesis supported by tiered structural evidence rather than as an empirically validated conclusion; its definitive validation requires evaluation infrastructure that does not yet exist in the field. Third, we document five pervasive evaluation failures (look-ahead bias, survivorship bias, backtesting overfitting, transaction cost neglect, and regime-shift blindness) and show that these can reverse the sign of reported returns. Building on the CPH and the evaluation critique, we introduce the Coordination Breakeven Spread (CBS), a metric for determining whether multi-agent coordination adds genuine value net of transaction costs, and propose minimum evaluation standards as prerequisites for validating the CPH.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。