让AI导航更稳更快,靠分阶段处理语言与空间信息
Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigation
- 用大模型理解指令转为2D坐标,再结合深度图转3D定位
- 导航成功率提升20%以上,路径误差减少约50%
- 适合需要稳定导航的无人机或机器人场景
当前视觉语言导航方法在语义理解与控制精度间存在权衡。虽多模态大语言模型(MLLM)具备更强推理能力,但将其作为底层控制器会导致高延迟、轨迹震荡和泛化差,源于几何定位不足。为此,我们提出Fly0,通过三阶段流程解耦语义推理与几何规划:(1) MLLM模块将自然语言指令转化为2D像素坐标;(2) 几何投影模块利用深度数据将目标定位至3D空间;(3) 几何规划器生成无碰撞路径。该机制可在视觉丢失时仍保持鲁棒导航。通过避免持续推理,Fly0降低计算开销并提升系统稳定性。仿真与真实环境实验表明,相比最先进基线,Fly0在非结构化环境中成功率达20%以上提升,导航误差(NE)降低约50%。代码已公开于https://github.com/xuzhenxing1/Fly0。
原文摘要 · Abstract (English)
Current Visual-Language Navigation (VLN) methodologies face a trade-off between semantic understanding and control precision. While Multimodal Large Language Models (MLLMs) offer superior reasoning, deploying them as low-level controllers leads to high latency, trajectory oscillations, and poor generalization due to weak geometric grounding. To address these limitations, we propose Fly0, a framework that decouples semantic reasoning from geometric planning. The proposed method operates through a three-stage pipeline: (1) an MLLM-driven module for grounding natural language instructions into 2D pixel coordinates; (2) a geometric projection module that utilizes depth data to localize targets in 3D space; and (3) a geometric planner that generates collision-free trajectories. This mechanism enables robust navigation even when visual contact is lost. By eliminating the need for continuous inference, Fly0 reduces computational overhead and improves system stability. Extensive experiments in simulation and real-world environments demonstrate that Fly0 outperforms state-of-the-art baselines, improving the Success Rate by over 20\% and reducing Navigation Error (NE) by approximately 50\% in unstructured environments. Our code is available at https://github.com/xuzhenxing1/Fly0.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。