提出新方法解决多智能体强化学习中奖励分配难题，确保学习效率与最优策略一致。
Redistributing Rewards Across Time and Agents for Multi-Agent Reinforcement Learning
- 用神经网络学贡献分，再通过确定性归一化保证总奖励不变
- 在SMACLite和GRF上加速学习，性能超越强基线模型
- 无需依赖模型精度，适合复杂协作任务的多智能体系统
信用分配，即分离每个智能体对共享奖励的贡献，是合作式多智能体强化学习（MARL）中的关键挑战。现有方法常通过强制返回等价性来保障，即分配奖励之和等于团队总奖励，但其有效性依赖于学习模型的回归精度，在实践中不可靠。本文提出时序-智能体奖励重分配（TAR$^2$），将信用建模与该约束解耦：神经网络学习未归一化的贡献得分，而独立的确定性归一化步骤天然满足返回等价性。我们证明该方法等价于一种有效的基于势函数的奖励塑造（PBRS），可保证无论模型精度如何，最优策略均被保留。在具有挑战性的SMACLite和Google Research Football（GRF）基准测试中，TAR$^2$显著加速学习并达到更高最终性能，验证了其在智能体-时间信用分配问题上的有效性。
原文摘要 · Abstract (English)
Credit assignmen, disentangling each agent's contribution to a shared reward, is a critical challenge in cooperative multi-agent reinforcement learning (MARL). To be effective, credit assignment methods must preserve the environment's optimal policy. Some recent approaches attempt this by enforcing return equivalence, where the sum of distributed rewards must equal the team reward. However, their guarantees are conditional on a learned model's regression accuracy, making them unreliable in practice. We introduce Temporal-Agent Reward Redistribution (TAR$^2$), an approach that decouples credit modeling from this constraint. A neural network learns unnormalized contribution scores, while a separate, deterministic normalization step enforces return equivalence by construction. We demonstrate that this method is equivalent to a valid Potential-Based Reward Shaping (PBRS), which guarantees the optimal policy is preserved regardless of model accuracy. Empirically, on challenging SMACLite and Google Research Football (GRF) benchmarks, TAR$^2$ accelerates learning and achieves higher final performance than strong baselines. These results establish our method as an effective solution for the agent-temporal credit assignment problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。