arXiv:2608.00635cs.RO2026-08

用流匹配联合预测未来场景和飞行轨迹,实现无人机实时自主导航。

FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation

论文配图:FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation
图 1 · 摘自论文原文
  • 通过双流Transformer共享注意力,同步优化深度图与飞行路径
  • 在8m/s高速下仍能稳定避障,真实飞行中延迟低于18ms
  • 适合对实时性要求高的无人机自主飞行系统

我们提出FlowPilot,一种基于深度图的轻量级世界-动作模型,用于无人机实时机载导航。不同于需局部重建或缺乏显式场景预测的端到端策略,FlowPilot利用流匹配联合去噪未来深度观测与可执行轨迹。双流混合Transformer通过共享注意力耦合视频与动作专家,使未来场景预测与轨迹生成相互促进。部署时模型以动作为中心,仅输出轨迹。为保证可追踪性,动作参数化为7次伯恩斯坦多项式:当前状态约束初始控制点,网络预测5个自由控制点,生成C^2连续参考轨迹,具备闭式速度、加速度和加加速度表达。模型在三级深度金字塔数据上训练:高吞吐仿真、拟真仿真与真实机载数据。闭环仿真中,在密集障碍物和最高8m/s指令速度下优于学习与优化基线。在物理四旋翼上,感知到动作全流程在Jetson Orin NX上耗时<18ms,于杂乱室内与林区环境达到5.5m/s速度,仅依赖机载传感与计算。

原文摘要 · Abstract (English)

We present FlowPilot, a compact world-action model for real-time onboard UAV navigation from depth. Unlike map-then-optimize pipelines that require local reconstruction or end-to-end policies that lack explicit scene prediction, FlowPilot jointly denoises future depth observations and executable trajectories with flow matching. A dual-stream mixture-of-transformers couples video and action experts through shared attention, allowing future-scene prediction and trajectory generation to inform each other. At deployment, the model runs action-centrically and outputs only a trajectory. To ensure trackability, actions are parameterized as degree-7 Bernstein polynomials: the current state constrains the initial control points, and the network predicts five free control points, yielding C^2-continuous references with closed-form velocity, acceleration and jerk. FlowPilot is trained on a three-level depth pyramid spanning high-throughput simulation, photorealistic simulation, and real onboard data. In closed-loop simulation, it outperforms learning- and optimization-based baselines under increasing clutter and commanded speeds up to 8m/s. On a physical quadrotor, the full perception-to-action pipeline runs in under 18ms on a Jetson Orin NX and reaches 5.5m/s in cluttered indoor and forest environments using only onboard sensing and computation.

无人机导航实时控制流匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。