arXiv:2510.14783cs.RO2025-10被引 9

首个端到端视觉无人机竞速系统,实现全自研、高鲁棒高速飞行。

SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learning

  • 基于模型强化学习,直接从像素生成电机指令,无需外部校准。
  • 实测最高21米/秒速度,6克加速度,可完成倒飞、急转弯等特技动作。
  • 自动估测电机极限,应对电池衰减,支持不同无人机快速部署。

自主无人机竞速(ADR)系统近年已达到冠军级表现,但高度依赖特定任务。尽管端到端视觉方法具有更广适用性潜力,至今尚无系统能同时实现完全的仿真到真实迁移、机载运行及冠军级性能。本文提出SkyDreamer,据我们所知首个直接从像素映射到电机命令的端到端视觉无人机竞速策略。其基于受控的Dreamer模型强化学习框架,世界模型在训练中解码仅训练期可用的特权信息。通过将该思想拓展至端到端视觉竞速,世界模型充当隐式状态与参数估计器,极大提升可解释性。SkyDreamer全程机载运行,无需外部辅助,利用世界模型隐藏状态追踪进展以解决视觉模糊问题,且无需外源相机标定,支持不同无人机快速部署而无需重训。真实实验表明,系统实现稳定高速飞行,完成倒飞、半滚倒转和梯形等复杂机动,最高速度达21米/秒,加速度高达6克。它还展现出非平凡的视觉仿真到真实迁移能力,在低质量分割掩码下运行,并通过实时估计最大电机转速应对电池耗尽,动态调整飞行路径。这些结果凸显SkyDreamer对现实差距关键因素的适应性,兼具高鲁棒性与极高速敏捷飞行性能。

原文摘要 · Abstract (English)

Autonomous drone racing (ADR) systems have recently achieved champion-level performance, yet remain highly specific to drone racing. While end-to-end vision-based methods promise broader applicability, no system to date simultaneously achieves full sim-to-real transfer, onboard execution, and champion-level performance. In this work, we present SkyDreamer, to the best of our knowledge, the first end-to-end vision-based ADR policy that maps directly from pixel-level representations to motor commands. SkyDreamer builds on informed Dreamer, a model-based reinforcement learning approach where the world model decodes to privileged information only available during training. By extending this concept to end-to-end vision-based ADR, the world model effectively functions as an implicit state and parameter estimator, greatly improving interpretability. SkyDreamer runs fully onboard without external aid, resolves visual ambiguities by tracking progress using the state decoded from the world model's hidden state, and requires no extrinsic camera calibration, enabling rapid deployment across different drones without retraining. Real-world experiments show that SkyDreamer achieves robust, high-speed flight, executing tight maneuvers such as an inverted loop, a split-S and a ladder, reaching speeds of up to 21 m/s and accelerations of up to 6 g. It further demonstrates a non-trivial visual sim-to-real transfer by operating on poor-quality segmentation masks, and exhibits robustness to battery depletion by accurately estimating the maximum attainable motor RPM and adjusting its flight path in real-time. These results highlight SkyDreamer's adaptability to important aspects of the reality gap, bringing robustness while still achieving extremely high-speed, agile flight.

无人机竞速端到端模型强化学习机载控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。