arXiv:2607.19809cs.MAcs.LG2026-07

用世界模型的隐状态实现多智能体长期信息通信,解决观测缺失时的协作难题。

Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning

论文配图:Dreamer-CPC: Message Learning with World Models for Decentralized Multi-agent Reinforcement Learning
图 1 · 摘自论文原文
  • 基于DreamerV3的世界模型,从历史隐状态中生成消息
  • 在观测缺失任务中,收益是基线方法的4到5倍
  • 适合需要长期记忆与跨智能体协作的场景

在多智能体强化学习中,智能体间通信能有效提升部分可观测环境下的性能。现有基于表示学习的方法虽能生成基于当前观测的消息,但无法传递时间累积的信息。本文提出Dreamer-CPC,将基于集体预测编码(CPC)的消息学习集成至DreamerV3的世界模型中。每个智能体独立维护世界模型和消息模块,从反映历史观测与动作的隐状态中推断并交换消息。在两个环境中评估:Observer(非合作信息共享任务)和CatchApple(新引入的任务,关键观测暂时缺失)。结果表明,Dreamer-CPC优于基于当前观测生成消息的IPPO-CPC及无通信基线,在CatchApple任务中,其每回合回报达IPPO-CPC的4至5倍,证明了基于世界模型隐动态的消息通信在当前观测不足时仍可实现有效协作。

原文摘要 · Abstract (English)

In multi-agent reinforcement learning (MARL), inter-agent communication is effective for improving performance under partial observability. Representation learning-based approaches enable decentralized agents to learn messages grounded in their own observations, but they rely only on current observations and cannot convey information accumulated over time. We propose Dreamer-CPC, a decentralized model-based MARL method that integrates message learning based on Collective Predictive Coding (CPC) into the world model of DreamerV3. Each agent independently maintains a world model and a message module, and infers and exchanges messages from the latent states of the world model that reflect the history of past observations and actions. We evaluated Dreamer-CPC in two environments: Observer, a non-cooperative information-sharing task, and CatchApple, a newly introduced task in which task-relevant observations are temporarily missing. In both environments, Dreamer-CPC outperformed IPPO-CPC, an existing CPC-based method that generates messages from current observations, as well as no-communication baselines. In particular, in CatchApple, Dreamer-CPC achieved 4 to 5 times the episode return of IPPO-CPC, demonstrating effective coordination where other methods fail due to missing observations. These results suggest that communication grounded in the latent dynamics of world models can support decentralized decision-making when current observations alone are insufficient.

多智能体世界模型通信机制强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。