arXiv:2410.15876cs.LGcs.AI2024-10

让多智能体强化学习应对动态增减的环境,提升泛化能力。

FlickerFusion: Intra-trajectory Domain Generalizing Multi-Agent RL

  • 通过随机丢弃观测信息模拟动态环境,增强模型鲁棒性。
  • 在未知环境下实现更高奖励,且不确定性显著降低。
  • 适用于搜救、作战等实体数量变化的真实场景。

多智能体强化学习在解决复杂协作任务方面展现出巨大潜力,但现有方法通常假设训练与推理时实体数量保持不变。这忽略了真实环境中实体可能动态增减的情况,如搜救任务或动态对抗场景。本文针对零样本跨域(OOD)泛化中的轨迹内动态实体组合问题,提出FlickerFusion——一种通用的MARL增强技术。该方法通过随机丢弃观测空间部分信息,使模型在推理时表现出域内特性。实验表明,相比现有方法,FlickerFusion不仅显著提升推理奖励,还有效降低不确定性。代码、模型权重及演示视频已开源,详见flickerfusion305.github.io。

原文摘要 · Abstract (English)

Multi-agent reinforcement learning has demonstrated significant potential in addressing complex cooperative tasks across various real-world applications. However, existing MARL approaches often rely on the restrictive assumption that the number of entities (e.g., agents, obstacles) remains constant between training and inference. This overlooks scenarios where entities are dynamically removed or added during the inference trajectory -- a common occurrence in real-world environments like search and rescue missions and dynamic combat situations. In this paper, we tackle the challenge of intra-trajectory dynamic entity composition under zero-shot out-of-domain (OOD) generalization, where such dynamic changes cannot be anticipated beforehand. Our empirical studies reveal that existing MARL methods suffer significant performance degradation and increased uncertainty in these scenarios. In response, we propose FlickerFusion, a novel OOD generalization method that acts as a universally applicable augmentation technique for MARL backbone methods. FlickerFusion stochastically drops out parts of the observation space, emulating being in-domain when inferenced OOD. The results show that FlickerFusion not only achieves superior inference rewards but also uniquely reduces uncertainty vis-à-vis the backbone, compared to existing methods. Benchmarks, implementations, and model weights are organized and open-sourced at flickerfusion305.github.io, accompanied by ample demo video renderings.

多智能体强化学习泛化能力动态环境

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。