让无人机听懂人话，智能穿越复杂城市环境。
SkyVLN: Vision-and-Language Navigation and NMPC Control for UAVs in Urban Environments
- 用大模型理解语言和视觉信息，实现精准导航
- 在新环境中导航成功率显著提升，支持回溯修正路径
- 结合动态避障控制，适合城市空域自主飞行任务
无人机在多个领域展现出强大应用潜力，其机动性和适应性尤为突出。本文提出SkyVLN框架，将视觉-语言导航（VLN）与非线性模型预测控制（NMPC）融合，提升无人机在复杂城市环境中的自主能力。不同于传统方法，SkyVLN利用大语言模型（LLMs）解析自然语言指令与视觉感知信息，使无人机能在动态三维空间中实现更高精度、更强鲁棒性的导航。我们设计了具备细粒度空间描述器和历史路径记忆机制的多模态导航代理，可消解空间歧义、应对模糊指令并实现必要回溯。框架还集成NMPC模块，实现动态障碍物规避，保障轨迹跟踪精度与防碰撞。为验证方法有效性，我们基于AirSim构建高保真3D城市仿真环境，包含真实图像与动态城市元素。大量实验表明，SkyVLN在未见环境中显著提升导航成功率与效率。
原文摘要 · Abstract (English)
Unmanned Aerial Vehicles (UAVs) have emerged as versatile tools across various sectors, driven by their mobility and adaptability. This paper introduces SkyVLN, a novel framework integrating vision-and-language navigation (VLN) with Nonlinear Model Predictive Control (NMPC) to enhance UAV autonomy in complex urban environments. Unlike traditional navigation methods, SkyVLN leverages Large Language Models (LLMs) to interpret natural language instructions and visual observations, enabling UAVs to navigate through dynamic 3D spaces with improved accuracy and robustness. We present a multimodal navigation agent equipped with a fine-grained spatial verbalizer and a history path memory mechanism. These components allow the UAV to disambiguate spatial contexts, handle ambiguous instructions, and backtrack when necessary. The framework also incorporates an NMPC module for dynamic obstacle avoidance, ensuring precise trajectory tracking and collision prevention. To validate our approach, we developed a high-fidelity 3D urban simulation environment using AirSim, featuring realistic imagery and dynamic urban elements. Extensive experiments demonstrate that SkyVLN significantly improves navigation success rates and efficiency, particularly in new and unseen environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。