用隐空间建模实现四轴飞行器零样本仿真到现实的长时序控制。
SkyJEPA: Learning Long-Horizon World Models for Zero-Shot Sim-to-Real Control of Quadrotors

- 基于隐空间动力学模型与物理启发探测器,实现长时预测。
- 在真实硬件上实现零样本迁移,户外闭环实验误差低于15%。
- 无需实测数据,自动化生成训练集,适合高动态飞行控制研究者。
精确的动力学模型对机器人系统在不确定环境下做出明智决策至关重要,尤其适用于高速空中飞行器。神经网络动力学模型能捕捉复杂非线性效应,但现有自回归预测方法在长时序预测中因误差累积而表现不佳。联合嵌入预测架构(JEPAs)通过在隐空间建模动力学提供了替代方案,但以往的JEPA方法多用于运动规划,未深入探索高频控制。本文提出一种适用于实时四轴飞行器控制的JEPA型模型,结合隐空间动力学模型与新型物理启发探测器,将冻结隐变量映射为可解释状态,实现物理合理的长时序预测。同时,将学习到的模型与基于采样的最优控制方案结合,充分发挥其预测能力,在嵌入式硬件上实现实时控制。为减少对昂贵且危险的真实数据采集的依赖,我们构建了自动化的数据集生成流程。大量开环与室外闭环实验表明,该方法具备高精度预测能力、强零样本仿真到现实迁移性能,并在多种工况下表现出良好泛化性。
原文摘要 · Abstract (English)
Accurate dynamics models are critical for informed decision-making in robotic systems, particularly for agile aerial vehicles operating under uncertainty. Neural network dynamics models are attractive for capturing complex nonlinear effects, but existing predictive approaches struggle with long-horizon forecasting because their autoregressive rollout mechanism amplifies errors over time. Joint Embedding Predictive Architectures (JEPAs) offer a compelling alternative by modeling dynamics in latent space, yet prior JEPA-style methods for robot navigation have been studied primarily for kinematic-level planning, with limited investigation in high-frequency control. In this work, we introduce the JEPA-style model for real-time quadrotor control. The proposed approach combines a latent dynamics model with a novel physics-inspired prober that maps frozen latents to interpretable state, enabling physically grounded long-horizon prediction. Additionally, we combine the learned model with a sampling-based optimal control solution to take advantage of its predictive capabilities for real-time control on embedded hardware. Finally, to reduce the dependence on expensive and unsafe real-world data collection, we develop a structured pipeline for automated dataset generation. Extensive open-loop and outdoor closed-loop experiments demonstrate accurate prediction, robust zero-shot sim-to-real transfer, and strong generalization across diverse operating conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。