构建可自进化细粒度记忆,提升零样本导航的准确性和泛化能力
EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigation

- 用视觉-语义图保存原始画面,按房间-视角-物体层级组织
- 压缩搜索空间后仅对关键区域调用大模型验证,效率更高
- 任务后自动更新经验知识,无需重训练即可优化后续决策
在零样本具身导航中,构建记忆对长程规划至关重要。以检测器为中心的场景图常将观测压缩为稀疏节点,丢失细粒度视觉证据并积累噪声;而基于3D重建的方法计算开销过大。本文提出EvoMemNav,一种高效、自进化的细粒度记忆框架。该框架构建视觉-语义记忆图(VSMGraph),将原始视图作为第一类记忆,结合轻量语义线索与拓扑关系,形成房间-视角-物体层级结构,保留细节用于消歧和停止验证。为应对持续增长的记忆,设计预算制粗到精策略:粗阶段压缩搜索空间至潜在区域,精阶段仅对目标区域调用视觉语言模型进行验证与决策。此外,EvoMemNav在每个子任务后执行反思式写回,更新附着于图的先验知识,编码环境累积经验,从而在不重训练的情况下优化未来决策。在GOAT-Bench与HM3D数据集上,针对物体、文本描述与图像目标模态的实验均显示成功率(SR)与路径相似率(SPL)一致提升,具备更强多实例消歧能力、更少提前停止,以及更优零样本泛化性能。
原文摘要 · Abstract (English)
Building memory is essential for long-horizon planning in zero-shot embodied navigation. Detector-centric scene graphs often compress observations into sparse nodes, discarding fine-grained visual evidence and accumulating noise, while 3D reconstruction-based methods remain computationally prohibitive. We present EvoMemNav, an efficient, self-evolving, fine-grained memory framework for zero-shot embodied navigation. EvoMemNav constructs a Visual-Semantic Memory Graph (VSMGraph) that keeps raw views as first-class memory and organizes them with lightweight semantic cues and topological relations into a room-view-object hierarchy, preserving fine-grained details for disambiguation and Stop verification. To scale to growing memory, we introduce a budgeted coarse-to-fine policy: a coarse stage compresses the search space into promising regions, and a fine stage invokes a VLM only for targeted verification and decision. Beyond static memories, EvoMemNav performs reflection-driven write-back after each subtask, updating graph-attached priors that encode accumulated environmental knowledge to refine future decisions without retraining. Experiments on GOAT-Bench and HM3D across object, text-description, and image-goal modalities show consistent gains in SR/SPL, with better multi-instance disambiguation, fewer premature stops, and stronger zero-shot generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。