用博弈论精准分配奖励，让多智能体系统更高效协作。
Who Deserves the Reward? SHARP: Shapley Credit-based Optimization for Multi-Agent System
- 基于谢林值计算每个智能体的边际贡献，实现精准奖励分配。
- 在多个真实场景中，比单智能体和多智能体方法平均提升23.66%和14.05%。
- 适合需要精细分工与协作的复杂任务系统，如工具调用、自动化决策。
将大语言模型与外部工具通过多智能体系统结合，为分解和解决复杂问题提供了新范式。然而，由于信用分配难题，训练这些系统仍极为困难——往往难以确定具体哪个功能智能体导致了决策轨迹的成功或失败。现有方法通常依赖稀疏或全局广播奖励，无法捕捉个体贡献，导致强化学习效率低下。为此，我们提出基于谢林值的分层归因强化策略（SHARP），一种通过精确信用分配优化多智能体强化学习的新框架。SHARP通过分解奖励机制有效稳定训练：包含全局广播-准确率奖励、每个智能体的谢林值边际信用奖励，以及提升执行效率的工具-流程奖励。在多个真实世界基准上的实验表明，SHARP显著优于近期最先进基线，在平均匹配度上分别比单智能体和多智能体方法提升23.66%和14.05%。
原文摘要 · Abstract (English)
Integrating Large Language Models (LLMs) with external tools via multi-agent systems offers a promising new paradigm for decomposing and solving complex problems. However, training these systems remains notoriously difficult due to the credit assignment challenge, as it is often unclear which specific functional agent is responsible for the success or failure of decision trajectories. Existing methods typically rely on sparse or globally broadcast rewards, failing to capture individual contributions and leading to inefficient reinforcement learning. To address these limitations, we introduce the Shapley-based Hierarchical Attribution for Reinforcement Policy (SHARP), a novel framework for optimizing multi-agent reinforcement learning via precise credit attribution. SHARP effectively stabilizes training by normalizing agent-specific advantages across trajectory groups, primarily through a decomposed reward mechanism comprising a global broadcast-accuracy reward, a Shapley-based marginal-credit reward for each agent, and a tool-process reward to improve execution efficiency. Extensive experiments across various real-world benchmarks demonstrate that SHARP significantly outperforms recent state-of-the-art baselines, achieving average match improvements of 23.66% and 14.05% over single-agent and multi-agent approaches, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。