用稀疏空间记忆实现零样本视觉导航,兼顾全局规划与局部避障。
MG-Nav: Dual-Scale Visual Navigation via Sparse Spatial Memory
- 构建区域中心的稀疏空间记忆图,融合多视角关键帧与物体语义。
- 在HM3D和MP3D上达到顶尖零样本性能,动态场景下仍稳定有效。
- 适合研究视觉导航、具身智能与跨场景泛化任务的开发者参考。
我们提出MG-Nav(Memory-Guided Navigation),一种统一全局记忆引导规划与局部几何增强控制的双尺度零样本视觉导航框架。核心是稀疏空间记忆图(SMG),一种紧凑的区域中心记忆结构,每个节点聚合多视角关键帧与物体语义,同时保留视角多样性。全局层面,代理在SMG上定位,并通过图像到实例的混合检索生成目标条件路径,输出一系列可达航点以实现长程引导。局部层面,导航基础策略以点目标模式执行航点,障碍物感知控制;从最后一个节点向视觉目标移动时切换至图像目标模式。为提升视角对齐与目标识别,引入VGGT-adapter,一个基于预训练VGGT模型的轻量级几何模块,将观测与目标特征映射到共享的3D感知空间。MG-Nav以不同频率运行全局规划与局部控制,并通过周期性重定位校正误差。在HM3D Instance-Image-Goal和MP3D Image-Goal基准上的实验表明,MG-Nav实现了当前最优的零样本表现,且在动态重排与未见场景条件下保持鲁棒性。
原文摘要 · Abstract (English)
We present MG-Nav (Memory-Guided Navigation), a dual-scale framework for zero-shot visual navigation that unifies global memory-guided planning with local geometry-enhanced control. At its core is the Sparse Spatial Memory Graph (SMG), a compact, region-centric memory where each node aggregates multi-view keyframe and object semantics, capturing both appearance and spatial structure while preserving viewpoint diversity. At the global level, the agent is localized on SMG and a goal-conditioned node path is planned via an image-to-instance hybrid retrieval, producing a sequence of reachable waypoints for long-horizon guidance. At the local level, a navigation foundation policy executes these waypoints in point-goal mode with obstacle-aware control, and switches to image-goal mode when navigating from the final node towards the visual target. To further enhance viewpoint alignment and goal recognition, we introduce VGGT-adapter, a lightweight geometric module built on the pre-trained VGGT model, which aligns observation and goal features in a shared 3D-aware space. MG-Nav operates global planning and local control at different frequencies, using periodic re-localization to correct errors. Experiments on HM3D Instance-Image-Goal and MP3D Image-Goal benchmarks demonstrate that MG-Nav achieves state-of-the-art zero-shot performance and remains robust under dynamic rearrangements and unseen scene conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。