提出无需近似即可精确评估多智能体大模型贡献的新方法
Exact Is Easier: Credit Assignment for Cooperative LLM Agents
- 通过固定交互历史实现决策点的精确回溯,直接计算无偏优势
- 在6个基准上超越所有基线,训练耗能降低且性能提升明显
- 首次实现不依赖特定方法的多智能体信用分配审计工具
在多智能体大模型系统中,移除一个智能体以评估其贡献看似合理,但此类评估会扭曲真实结果。传统方法依赖近似估计,源于对环境特权访问的错误假设。但在协作式大模型系统中,交互历史是可观测文本的确定性函数,无隐藏状态,因此每个决策点均可精确恢复,使直接因果测量成为可能。C3 方法通过在每个决策点固定完整历史,冻结行为策略并采样替代动作,以无参留一法基线计算每步无偏优势。在涵盖数学推理与代码生成的六个基准、两种模型族和两种多智能体拓扑上,C3 均显著优于所有基线;控制分解证实收益源于信用质量而非架构设计,且检查点恢复降低了训练令牌消耗。精确解比所有近似方案更简单、更低成本、更有效。该结构特性也支持精确验证:三个独立可计算诊断(信用保真度、组内方差、跨智能体影响)构成首个方法无关的多智能体信用分配审计工具。代码已开源。
原文摘要 · Abstract (English)
Removing an agent from a cooperative team to measure its contribution seems natural, yet in multi-agent LLM systems this evaluation distorts the result it claims to measure. This failure is not isolated: learned critics, trajectory-level baselines, and agent-removal counterfactuals all inherit from standard multi-agent reinforcement learning a premise that exact counterfactual evaluation requires privileged environment access, and therefore approximate. In cooperative LLM systems, this premise is false. Interaction histories are deterministic functions of observable text with no hidden state, so any decision point can be restored exactly, making direct causal measurement possible without parametric approximation. C3 exploits this property by fixing the complete history at each decision point, sampling alternative actions under a frozen behavior policy, and computing unbiased per-decision advantages through a parameter-free leave-one-out baseline. Across six benchmarks spanning math reasoning and code generation, two model families, and two multi-agent topologies, C3 consistently outperforms all baselines; a controlled decomposition confirms gains originate from credit quality, not architecture, while checkpoint restoration reduces training token consumption. The exact solution proves simpler, cheaper, and more effective than all approximate alternatives. The same structural property that enables exact credit also enables exact verification: three independently computable diagnostics, credit fidelity, within-group variance, and inter-agent influence, constitute the first method-agnostic auditing tool for multi-agent LLM credit assignment. Our code is available at https://github.com/EIT-EAST-Lab/C3
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。