一个神经网络能控制多种无人机,无需重新训练。
RAPTOR: A Foundation Policy for Quadrotor Control
- 用元模仿学习训练通用无人机控制策略,支持零样本适配。
- 仅2084参数的三层网络即可控制32克至2.4公斤的10种不同无人机。
- 毫秒级适应新机型,适合需快速部署的机器人系统研发者。
人类在面对新环境时表现出极高的数据效率,例如驾驶新车。相比之下,现代机器人控制系统(如强化学习训练的神经网络策略)通常针对单一环境高度定制,容易因微小差异(如仿真到现实的差距)而失效,且需重新识别和训练才能应对系统变化。本文提出RAPTOR,一种用于四旋翼飞行器控制的通用基础策略。该方法训练单个端到端神经网络策略,可控制多种四旋翼平台。我们在10种真实四旋翼上进行测试,重量范围从32克到2.4公斤,涵盖刷式/无刷电机、软/刚性机架、2/3/4桨叶以及不同飞控系统(PX4/Betaflight/Crazyflie/M5StampFly)。结果表明,一个仅有2084参数的三层策略即可实现零样本适配。通过隐藏层中的循环结构实现上下文学习。策略采用提出的元模仿学习算法训练:先为1000种不同四旋翼各训练一个教师策略(使用强化学习),再将这些教师模型蒸馏成一个统一的自适应学生策略。结果显示,该基础策略可在毫秒内实现对未见过四旋翼的零样本适应。我们还在多种条件下全面测试其性能(轨迹跟踪、室内外飞行、风扰动、物理触碰、不同螺旋桨)。
原文摘要 · Abstract (English)
Humans are remarkably data-efficient when adapting to new unseen conditions, like driving a new car. In contrast, modern robotic control systems, like neural network policies trained using Reinforcement Learning (RL), are highly specialized for single environments. Because of this overfitting, they are known to break down even under small differences like the Simulation-to-Reality (Sim2Real) gap and require system identification and retraining for even minimal changes to the system. In this work, we present RAPTOR, a method for training a highly adaptive foundation policy for quadrotor control. Our method enables training a single, end-to-end neural-network policy to control a wide variety of quadrotors. We test 10 different real quadrotors from 32 g to 2.4 kg that also differ in motor type (brushed vs. brushless), frame type (soft vs. rigid), propeller type (2/3/4-blade), and flight controller (PX4/Betaflight/Crazyflie/M5StampFly). We find that a tiny, three-layer policy with only 2084 parameters is sufficient for zero-shot adaptation to a wide variety of platforms. The adaptation through in-context learning is made possible by using a recurrence in the hidden layer. The policy is trained through our proposed Meta-Imitation Learning algorithm, where we sample 1000 quadrotors and train a teacher policy for each of them using RL. Subsequently, the 1000 teachers are distilled into a single, adaptive student policy. We find that within milliseconds, the resulting foundation policy adapts zero-shot to unseen quadrotors. We extensively test the capabilities of the foundation policy under numerous conditions (trajectory tracking, indoor/outdoor, wind disturbance, poking, different propellers).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。