通过多智能体竞争赛车，让无人机自动学会高速飞行与策略博弈。
Agile Flight Emerges from Multi-Agent Competitive Racing
- 用胜负作为稀疏奖励，让多个智能体在竞争中自发产生敏捷飞行能力。
- 在有障碍物的复杂环境中，性能优于传统单智能体逐点奖励方法。
- 训练出的策略更易迁移到真实世界，且能应对未见过的对手。
通过多智能体竞争和获胜这一稀疏的高层目标，我们发现基于强化学习训练的智能体能够自发涌现出敏捷飞行（如高速运动逼近平台物理极限）和策略行为（如超车或阻拦）。我们在仿真和真实世界中均证明，该方法在环境复杂度增加（如存在障碍物）时，显著优于传统单智能体训练方式（如以赛道进度为奖励）。此外，尽管两种方法使用相同的仿真环境、随机化策略和硬件，多智能体竞争训练出的策略仍具备更强的模拟到现实的迁移能力。同时，这些策略对训练时未见过的对手也展现出一定泛化能力。本工作延续数字领域多智能体竞技的传统，表明仅用任务级稀疏奖励即可训练出具备高级低层控制能力的真实世界智能体。
原文摘要 · Abstract (English)
Through multi-agent competition and the sparse high-level objective of winning a race, we find that both agile flight (e.g., high-speed motion pushing the platform to its physical limits) and strategy (e.g., overtaking or blocking) emerge from agents trained with reinforcement learning. We provide evidence in both simulation and the real world that this approach outperforms the common paradigm of training agents in isolation with rewards that prescribe behavior, e.g., progress on the raceline, in particular when the complexity of the environment increases, e.g., in the presence of obstacles. Moreover, we find that multi-agent competition yields policies that transfer more reliably to the real world than policies trained with a single-agent progress-based reward, despite the two methods using the same simulation environment, randomization strategy, and hardware. In addition to improved sim-to-real transfer, the multi-agent policies also exhibit some degree of generalization to opponents unseen at training time. Overall, our work, following in the tradition of multi-agent competitive game-play in digital domains, shows that sparse task-level rewards are sufficient for training agents capable of advanced low-level control in the physical world. Code: https://github.com/Jirl-upenn/AgileFlight_MultiAgent
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。