让智能体学会稳健合作，避免搭便车，与新伙伴也能默契配合。
Training Generalizable Collaborative Agents via Strategic Risk Aversion
- 引入战略风险规避机制，提升合作策略的鲁棒性。
- 在协作博弈中均衡表现优于纳什均衡，且减少搭便车现象。
- 适用于大模型协作等复杂场景，适合需要跨伙伴泛化的任务。
许多新兴的智能体范式要求智能体之间（或与人）协作以实现共同目标。然而，现有协作策略学习方法产生的解决方案在面对新伙伴时容易失效。我们归因于训练中的搭便车行为和缺乏战略鲁棒性。为此，我们研究了战略风险规避的概念，并将其视为面向未知伙伴泛化合作的合理归纳偏置。具有战略风险规避特性的参与者天生对伙伴行为偏差具有鲁棒性，我们发现其在协作博弈中（1）可获得优于经典博弈论概念（如纳什均衡）的均衡结果，（2）表现出更少甚至无搭便车行为。受此启发，我们开发了一种将战略风险规避融入标准策略优化的多智能体强化学习（MARL）算法。在多个协作基准测试（包括一个大型语言模型协作任务）上的实证结果验证了理论，并表明该方法在不同任务中能稳定地与异构且此前未见过的伙伴实现可靠协作。
原文摘要 · Abstract (English)
Many emerging agentic paradigms require agents to collaborate with one another (or people) to achieve shared goals. Unfortunately, existing approaches to learning policies for such collaborative problems produce brittle solutions that fail when paired with new partners. We attribute these failures to a combination of free-riding during training and a lack of strategic robustness. To address these problems, we study the concept of strategic risk aversion and interpret it as a principled inductive bias for generalizable cooperation with unseen partners. While strategically risk-averse players are robust to deviations in their partner's behavior by design, we show that, in collaborative games, they also (1) can have better equilibrium outcomes than those at classical game-theoretic concepts like Nash, and (2) exhibit less or no free-riding. Inspired by these insights, we develop a multi-agent reinforcement learning (MARL) algorithm that integrates strategic risk aversion into standard policy optimization methods. Our empirical results across collaborative benchmarks (including an LLM collaboration task) validate our theory and demonstrate that our approach consistently achieves reliable collaboration with heterogeneous and previously unseen partners across collaborative tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。