用带标签语义图替代历史图像,提升视觉语言导航效率
MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation
- 用顶视语义图+文本标注构建可读导航记忆
- 在仿真和真实环境均达当前最优性能
- 适合关注高效导航记忆设计的研究者
视觉-语言导航(VLN)是具身智能中的关键任务,要求智能体在多样且未见的环境中遵循自然语言指令进行导航。传统方法依赖历史观测作为时空上下文,导致存储与计算开销巨大。本文提出MapNav,一种全新的端到端VLN模型,通过引入带标注的语义地图(ASM)替代历史帧。具体而言,模型在每轮开始时构建俯视语义地图,并在每个时间步更新,实现精确物体定位与结构化导航信息。随后,通过为关键区域添加显式文本标签,将抽象语义转化为清晰导航线索,生成最终的ASM。MapNav以构建的ASM为输入,利用视觉语言模型的强大端到端能力实现导航。大量实验表明,MapNav在仿真与真实场景中均达到当前最优表现,验证了该方法的有效性。此外,我们将开源ASM生成代码与数据集,保障可复现性,为领域提供重要资源。我们相信MapNav可作为VLN中新的记忆表示方法,推动该方向未来发展。
原文摘要 · Abstract (English)
Vision-and-language navigation (VLN) is a key task in Embodied AI, requiring agents to navigate diverse and unseen environments while following natural language instructions. Traditional approaches rely heavily on historical observations as spatio-temporal contexts for decision making, leading to significant storage and computational overhead. In this paper, we introduce MapNav, a novel end-to-end VLN model that leverages Annotated Semantic Map (ASM) to replace historical frames. Specifically, our approach constructs a top-down semantic map at the start of each episode and update it at each timestep, allowing for precise object mapping and structured navigation information. Then, we enhance this map with explicit textual labels for key regions, transforming abstract semantics into clear navigation cues and generate our ASM. MapNav agent using the constructed ASM as input, and use the powerful end-to-end capabilities of VLM to empower VLN. Extensive experiments demonstrate that MapNav achieves state-of-the-art (SOTA) performance in both simulated and real-world environments, validating the effectiveness of our method. Moreover, we will release our ASM generation source code and dataset to ensure reproducibility, contributing valuable resources to the field. We believe that our proposed MapNav can be used as a new memory representation method in VLN, paving the way for future research in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。