用分层记忆让机器人零样本导航更省力更准
HAM-VLN: Harnessing Hierarchical Agentic Memory for Zero-Shot Vision-and-Language Navigation
- 让机器人每步决策时自动记录环境语义和反思信息
- 在不训练情况下，三类任务成功率超60%，上下文长度减少65%以上
- 适合研究零样本视觉语言导航与高效记忆设计的读者
视觉-语言导航（VLN）使机器人能在未见过的环境中遵循指令行动。近期出现无需训练的范式：机器人通过多模态大模型理解观测并规划下一步动作。但基于图像流或密集地图的长程导航会带来持续增长的记忆与推理瓶颈。本文提出HAM-VLN，一种决策耦合、由智能体自动生成的记忆系统，为机器人构建持久且深度感知的世界图。在每一步决策的模型调用中，系统同时记录语义与反思信息——包括房间类型、物体、导航进度和失败记录。近期路径点保持原样在有限窗口内，旧历史仅通过相关性、时效性和显著性评分检索，并结合单跳拓扑扩展重新进入上下文。该设计无需额外大模型调用，仅依赖每一步决策。相比先前方法，HAM-VLN不仅提升各类导航指标，还使上下文长度减少超过65%。具体表现：在VLN-CE R2R上达到61.0%成功率（SR），VLN-CE RxR上52.7% SR，HM3D-v2 ObjectNav上79.7% SR，均无需任何训练。
原文摘要 · Abstract (English)
Vision-and-language navigation (VLN) enables robots to follow instructions in previously unseen environments. Recently, a training-free paradigm has emerged: the robot queries a multimodal LLM to understand its observations and plan the next action. However, long-horizon navigation based on either image streams or dense map inevitably introduces a growing memory and reasoning bottleneck. We present HAM-VLN, a decision-coupled, agent-authored memory that equips the robot with a persistent, depth-grounded world graph. In the same model call used to select the next action, HAM-VLN also records semantic and reflective information---including room type, objects, navigation progress, and failure notes. Recent waypoints remain verbatim within a bounded window, while older history re-enters the context only through retrieval scored by relevance, recency, and salience, together with one-hop topological expansion. This design requires no additional LLM calls beyond the per-waypoint decision. Compared to previous methods, HAM-VLN not only improves various navigation metrics but also reduces the context length by more than 65%. Specifically, HAM-VLN achieves 61.0% Success Rate (SR) on VLN-CE R2R, 52.7% SR on VLN-CE RxR, and 79.7% SR on HM3D-v2 ObjectNav without any training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。