通过观察同伴动态校准好奇心，让多智能体在稀疏奖励下更高效探索。
Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibration
- 基于同伴行为动态实时校准内在好奇心，过滤噪声干扰。
- 在VMAS、Meltingpot等基准上显著超越现有最优算法。
- 适合通信受限的分布式多智能体强化学习场景。
在稀疏奖励的复杂多智能体强化学习中，自主探索依赖于有效的内在动机。尽管人工好奇心提供强大的自监督信号，但常将环境随机性误认为有意义的新颖性。现有机制存在统一的新颖性偏见，对所有意外观测一视同仁。而同伴行为所蕴含的任务动态信息常被忽略，导致去中心化无通信设置下的探索效率低下。受人类儿童通过观察同伴调节探索行为的启发，我们提出CERMIC：一种基于多智能体上下文动态校准内在好奇心的框架。该方法能鲁棒地过滤噪声惊喜信号，并生成理论支持的内在奖励，引导智能体探索高信息增益的状态转移。在VMAS、Meltingpot和SMACv2等基准上的实验证明，CERMIC在稀疏奖励环境中显著优于现有最先进算法。
原文摘要 · Abstract (English)
Autonomous exploration in complex multi-agent reinforcement learning (MARL) with sparse rewards critically depends on providing agents with effective intrinsic motivation. While artificial curiosity offers a powerful self-supervised signal, it often confuses environmental stochasticity with meaningful novelty. Moreover, existing curiosity mechanisms exhibit a uniform novelty bias, treating all unexpected observations equally. However, peer behavior novelty, which encode latent task dynamics, are often overlooked, resulting in suboptimal exploration in decentralized, communication-free MARL settings. To this end, inspired by how human children adaptively calibrate their own exploratory behaviors via observing peers, we propose a novel approach to enhance multi-agent exploration. We introduce CERMIC, a principled framework that empowers agents to robustly filter noisy surprise signals and guide exploration by dynamically calibrating their intrinsic curiosity with inferred multi-agent context. Additionally, CERMIC generates theoretically-grounded intrinsic rewards, encouraging agents to explore state transitions with high information gain. We evaluate CERMIC on benchmark suites including VMAS, Meltingpot, and SMACv2. Empirical results demonstrate that exploration with CERMIC significantly outperforms SoTA algorithms in sparse-reward environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。