用物理模型提升无人机强化学习的泛化能力
Dreaming Falcon: Physics-Informed Model-Based Reinforcement Learning for Quadcopters
- 将无人机建模为自由体，通过物理动力学预测受力与运动
- 在训练数据上表现良好，但传统RNN模型难以泛化到新轨迹
- 适合关注无人机控制鲁棒性的研究者和工程师
当前飞行器控制算法在动态环境和恶劣条件下的鲁棒性不足。基于模型的强化学习（RL）虽具潜力且样本高效，但将Dreamer类方法应用于空中系统仍面临挑战，主要源于样本效率低及动力学模型泛化能力差。本文提出一种物理信息引导的世界模型学习方法：将四旋翼无人机视为自由体系统，直接预测其受到的净力与力矩，并输入6-自由度的龙格-库塔积分器（RK4）以生成未来状态的滚动预测。实验对比了该物理信息方法与标准RNN世界模型的表现。两者在训练数据上均表现良好，但在新轨迹上均出现状态预测快速发散，导致策略无法收敛。
原文摘要 · Abstract (English)
Current control algorithms for aerial robots struggle with robustness in dynamic environments and adverse conditions. Model-based reinforcement learning (RL) has shown strong potential in handling these challenges while remaining sample-efficient. Additionally, Dreamer has demonstrated that online model-based RL can be achieved using a recurrent world model trained on replay buffer data. However, applying Dreamer to aerial systems has been quite challenging due to its sample inefficiency and poor generalization of dynamics models. Our work explores a physics-informed approach to world model learning and improves policy performance. The world model treats the quadcopter as a free-body system and predicts the net forces and moments acting on it, which are then passed through a 6-DOF Runge-Kutta integrator (RK4) to predict future state rollouts. In this paper, we compare this physics-informed method to a standard RNN-based world model. Although both models perform well on the training data, we observed that they fail to generalize to new trajectories, leading to rapid divergence in state rollouts, preventing policy convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。