用玻尔兹曼探索提升多智能体规划的鲁棒性
Boltzmann-based Exploration for Robust Decentralized Multi-Agent Planning (Extended Version)
- 用玻尔兹曼策略替代确定性选择，结合熵奖励持续探索
- 在欺骗性场景中优于传统方法，标准任务上保持竞争力
- 适合需要稳定探索的分布式多智能体系统
去中心化蒙特卡洛树搜索（Dec-MCTS）广泛用于协作式多智能体规划，但在稀疏或偏斜奖励环境中表现不佳。本文提出协同玻尔兹曼蒙特卡洛树搜索（CB-MCTS），将确定性的UCT策略替换为随机的玻尔兹曼策略，并引入衰减的熵奖励以实现持续且集中的探索。尽管玻尔兹曼探索已在单智能体MCTS中研究，但其在多智能体系统中的应用面临独特挑战。CB-MCTS是首个解决此问题的方法。我们在简单后悔率设置下分析了该方法，并通过模拟验证其在欺骗性场景中优于Dec-MCTS，同时在标准基准测试上保持竞争力，为多智能体规划提供了稳健解决方案。
原文摘要 · Abstract (English)
Decentralized Monte Carlo Tree Search (Dec-MCTS) is widely used for cooperative multi-agent planning but struggles in sparse or skewed reward environments. We introduce Coordinated Boltzmann MCTS (CB-MCTS), which replaces deterministic UCT with a stochastic Boltzmann policy and a decaying entropy bonus for sustained yet focused exploration. While Boltzmann exploration has been studied in single-agent MCTS, applying it in multi-agent systems poses unique challenges. CB-MCTS is the first to address this. We analyze CB-MCTS in the simple-regret setting and show in simulations that it outperforms Dec-MCTS in deceptive scenarios and remains competitive on standard benchmarks, providing a robust solution for multi-agent planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。