arXiv:2606.12281cs.MAcs.AI2026-06

让智能体通过共识机制更聪明地吸收导师建议,提升协作效率。

CCKS: Consensus-based Communication and Knowledge Sharing

论文配图:CCKS: Consensus-based Communication and Knowledge Sharing
图 1 · 摘自论文原文
  • 基于对比学习构建共识模型,动态评估教师指导的适用性
  • 在足球和星际争霸环境中,合作效率与学习速度显著提升
  • 可直接接入现有算法,适合多智能体强化学习研究者

在去中心化训练与执行(DTDE)的多智能体强化学习中,基于动作建议的知识共享能实现可解释且可扩展的协作。然而,现有方法过度依赖教师指导而缺乏对师生适配性的评估,导致过度建议、稳定性差和性能下降。为此,本文提出一种基于共识的通信与知识共享(CCKS)框架,使智能体根据共识约束选择是否采纳建议,更智能地平衡探索与学习。核心在于利用对比学习,基于智能体本地观测构建共识模型。在动作选择时,智能体结合共识与共享知识进行评分并决策。作为即插即用方案,CCKS可无缝集成至现有DTDE算法。在Google Research Football环境及复杂的StarCraft II Multi-Agent Challenge中的实验表明,引入CCKS后,协作效率、学习速度与整体性能均显著优于现有基线。代码已开源:https://github.com/yuanxpy/CCKS。

原文摘要 · Abstract (English)

In Decentralized Training and Decentralized Execution (DTDE) for cooperative Multi-Agent Reinforcement Learning (MARL), action-advising-based knowledge sharing promotes interpretable and scalable cooperation among agents. However, current action advising approaches often adhere too much to the teacher's guidance without evaluating teacher-student compatibility, which causes excessive advising, suboptimal stability, and degraded performance. To overcome these challenges, this paper presents a Consensus-based Communication and Knowledge Sharing (CCKS) framework, which allows agents to adopt recommendations based on consensus-derived constraints and to follow the teacher's instructions more smartly. This mechanism enables agents to balance exploration and learning from experienced teachers, improving overall performance. The key is the consensus model construction, for which we propose to employ contrastive learning to construct consensus models based on local observations in the agents' training phase. In action selection, agents score and choose actions based on consensus and shared knowledge. Designed as a plug-and-play solution, CCKS integrates seamlessly with existing DTDE algorithms. Experiments conducted in the Google Research Football environment and the complex StarCraft II Multi-Agent Challenge demonstrate that the integration with CCKS significantly improves cooperation efficiency, learning speed, and overall performance compared with current DTDE baselines. The code is available at https://github.com/yuanxpy/CCKS.

多智能体强化学习知识共享共识机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。