arXiv:2512.08639cs.CVcs.AI2025-12被引 8

仅用单目视觉和语言指令,让无人机在城市中自主导航。

Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning

  • 将导航任务转为文本预测,统一处理空间、时间与动作推理。
  • 单目RGB下表现超越现有基线,接近全景+深度的先进方法。
  • 适合轻量级无人机部署,减少传感器依赖,降低系统成本。

航拍视觉-语言导航旨在使无人机通过机载视觉观测和自然语言指令,在复杂城市环境中自主导航,具有低空巡检、搜救及自主空投等实际应用前景。现有方法通常依赖全景图像、深度信息或里程计支持空间推理与路径规划,增加系统成本与集成复杂度,不利于轻量化无人机的实际部署。本文提出一种仅基于自视角单目RGB图像与自然语言指令的统一航拍VLN框架。模型将导航建模为下一步文本预测问题,通过提示引导的多任务学习联合优化空间感知、轨迹推理与动作预测。此外,设计关键帧选择策略以保留语义信息丰富的帧,减少视觉冗余;提出动作合并与标签重加权机制,缓解长尾监督不平衡问题,促进稳定多任务协同训练。在AerialVLN与OpenFly基准上的大量实验表明,该方法在仅使用单目RGB的挑战性设置下,于已见与未见环境均取得优异性能,显著优于现有仅用RGB的基线,并缩小了与顶尖全景RGB-D方法之间的差距。全面消融实验进一步验证了任务设计与架构选择的有效性。代码已公开于https://github.com/return-sleep/AeroAct。

原文摘要 · Abstract (English)

Aerial Vision-and-Language Navigation (VLN) aims to enable unmanned aerial vehicles (UAVs) to interpret natural language instructions and navigate complex urban environments using onboard visual observation. This task holds promise for real-world applications such as low-altitude inspection, search-and-rescue, and autonomous aerial delivery. Existing methods often rely on panoramic images, depth inputs, or odometry to support spatial reasoning and action planning. These requirements increase system cost and integration complexity, thus hindering practical deployment for lightweight UAVs. We present a unified aerial VLN framework that operates solely on egocentric monocular RGB observations and natural language instructions. The model formulates navigation as a next-token prediction problem, jointly optimizing spatial perception, trajectory reasoning, and action prediction through prompt-guided multi-task learning. Moreover, we propose a keyframe selection strategy to reduce visual redundancy by retaining semantically informative frames, along with an action merging and label reweighting mechanism that mitigates long-tailed supervision imbalance and facilitates stable multi-task co-training. Extensive experiments on the AerialVLN and OpenFly benchmark validate the effectiveness of our method. Under the challenging monocular RGB-only setting, our model achieves strong results across both seen and unseen environments. It significantly outperforms existing RGB-only baselines and narrows the performance gap with state-of-the-art panoramic RGB-D counterparts. Comprehensive ablation studies further demonstrate the contribution of our task design and architectural choices. Our code is publicly available at https://github.com/return-sleep/AeroAct.

无人机导航视觉语言单目视觉多任务学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。