arXiv:2505.05622cs.ROcs.AI2025-05ACL被引 49

用分层语义规划与全局记忆,让无人机听懂指令飞遍城市

CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory

  • 分层语义规划将长任务拆解为多级子目标,降低导航复杂度
  • 在连续城市环境中实现当前最优表现,显著提升导航成功率
  • 适合研究空中视觉语言导航与智能体路径规划的开发者

空中视觉-语言导航(Aerial VLN)要求无人机理解自然语言指令并在复杂城市环境中导航,是连接人机交互、三维空间推理与真实部署的关键挑战。尽管现有地面VLN智能体在室内外场景取得显著成果,但在空中场景因缺乏预定义导航图和动作空间指数级扩张而表现不佳。本文提出CityNavAgent,一种基于大语言模型(LLM)的智能体,显著降低城市空中导航的复杂性。设计分层语义规划模块(HSPM),将长程任务分解为不同语义层级的子目标,通过不同能力级别的LLM逐步达成。同时引入全局记忆模块,将历史轨迹存储为拓扑图,简化已访问目标的导航。大量基准实验表明,该方法在连续城市环境中达到当前最优性能,且模块有效性得到验证。代码已开源。

原文摘要 · Abstract (English)

Aerial vision-and-language navigation (VLN), requiring drones to interpret natural language instructions and navigate complex urban environments, emerges as a critical embodied AI challenge that bridges human-robot interaction, 3D spatial reasoning, and real-world deployment. Although existing ground VLN agents achieved notable results in indoor and outdoor settings, they struggle in aerial VLN due to the absence of predefined navigation graphs and the exponentially expanding action space in long-horizon exploration. In this work, we propose \textbf{CityNavAgent}, a large language model (LLM)-empowered agent that significantly reduces the navigation complexity for urban aerial VLN. Specifically, we design a hierarchical semantic planning module (HSPM) that decomposes the long-horizon task into sub-goals with different semantic levels. The agent reaches the target progressively by achieving sub-goals with different capacities of the LLM. Additionally, a global memory module storing historical trajectories into a topological graph is developed to simplify navigation for visited targets. Extensive benchmark experiments show that our method achieves state-of-the-art performance with significant improvement. Further experiments demonstrate the effectiveness of different modules of CityNavAgent for aerial VLN in continuous city environments. The code is available at \href{https://github.com/VinceOuti/CityNavAgent}{link}.

视觉语言导航无人机大模型路径规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。