提升多智能体路径规划在观测攻击下的鲁棒性,无需改变部署方式。
Robust Multi-Agent Path Finding under Observation Attacks: A Principled Adversarial-Plus-Smoothing Training Recipe
- 通过对抗训练和平滑优化,增强策略对输入扰动的抵抗力。
- 在8×8地图4智能体场景下,抗攻击成功率从2.5%提升至77.5%。
- 适用于需要高可靠性的分布式路径规划系统,如无人机编队。
去中心化多智能体路径规划(MAPF)在共享网格上调度一组智能体,每个智能体基于自身局部观测行动。标准方法使用近端策略优化(PPO)训练一个共享神经策略,该策略在干净观测下表现良好,但单个智能体的微小输入扰动常导致其动作变化,进而阻塞邻近智能体,造成团队堵塞。本文提出两种训练方案,在不改变网络结构与部署流程的前提下,使策略在观测受扰时仍能稳定运行。第一种方案Adv-PPO在最坏情况扰动下训练共享策略,并选择对抗鲁棒性最优的检查点;第二种方案Adv-PPO+MACER在此基础上引入小规模的在线策略平滑项,其梯度遵循随机平滑的认证半径。在POGEMA数据集的8×8地图与四智能体设置中,未防护的PPO策略在干净条件下成功率达95.8%,但在最强攻击下仅2.5%。Adv-PPO将最坏情况成功率恢复至59.2%,清洁性能下降1个百分点;Adv-PPO+MACER在三次独立种子测试中达到77.5%±6.0%,清洁性能损失不足1个百分点。实验包含各攻击强度下的成功率曲线、经认证的动作稳定性检验(针对平滑策略包装器,非部署的argmax策略),以及同一环境实例中的失败模式与修复对比可视化故事板。
原文摘要 · Abstract (English)
Decentralized multi-agent path finding (MAPF) routes a team of agents on a shared grid, each acting from its own local view. The standard solution trains one shared neural policy with Proximal Policy Optimization (PPO), a popular on-policy reinforcement learning algorithm. Such a policy works well on clean observations, but a small input perturbation on one agent often changes its action, which then blocks a neighbour, and the team jams. In this paper we present two training recipes that keep the same network and the same deployment loop, yet make the policy hold up under perturbed observations. The first recipe, Adv-PPO, trains the shared policy against worst-case perturbations of its own input and selects the checkpoint by performance under adversarial perturbation. The second recipe, Adv-PPO+MACER, fine-tunes that checkpoint with a small on-policy smoothness term whose gradient follows the certified radius of randomized smoothing. On POGEMA with 8x8 maps and four agents, the unprotected PPO policy reaches 95.8% clean success but only 2.5% under the strongest attack. Adv-PPO recovers worst-case success to 59.2% at one percentage point of clean cost. Adv-PPO+MACER recovers it to 77.5% +/- 6.0% across three independent seeds at less than one percentage point of clean cost. We support these numbers with per-attack curves, a certified action-stability sanity check (which measures the smoothed-policy wrapper, not the deployed argmax policy), and side-by-side rollout storyboards that show the failure mode and the fix inside one environment instance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。