让视觉语言模型更高效地导航,通过2D选点+动态记忆+分层对齐实现
Embodied-Navigator: Point, Think, Memorize, and Align for Efficient Navigation

- 用2D像素选择代替复杂动作,自然对接模型视觉能力
- 关键节点存记忆、冗余轨迹压缩为轻量指标,提升时空感知
- 分层奖励机制让思考与环境反馈实时对齐,适合高效训练
尽管大视觉语言模型(VLMs)显著推进了具身导航,但直接部署仍面临挑战:现有方法常迫使VLM进入不自然的动作空间,违背其2D预训练先验,且推理流程僵化、记忆管理低效。为此,我们提出TAMP-Nav统一框架。首先,引入像素到3D动作映射(Point),将导航转化为2D视觉提示:VLM仅需选择2D像素,再投影至3D坐标供底层SLAM控制器执行,天然契合模型2D视觉能力。其次,设计集成的可选推理与锚点-轨迹记忆机制(Think and Memorize),动态触发思维链,并仅在关键节点保留高保真记忆,将冗余轨迹压缩为轻量级时空指标,有效保存历史信息并增强时空感知。最后,通过分组相对策略优化(GRPO)构建双层对齐范式(Align),叠加全局结果奖励与细粒度过程奖励,实现密集监督,使模型认知规划与物理环境反馈紧密对齐,赋予自适应推理能力。实验表明,TAMP-Nav在R2R-CE上达到66.2%成功率,具备高运行效率与样本效率(仅需90k训练轨迹)。
原文摘要 · Abstract (English)
Although Large Vision-Language Models (VLMs) have significantly advanced embodied navigation, their direct deployment remains challenging, as existing methods often force VLMs into unnatural action spaces that misalign with their 2D pre-training priors, compounded by rigid reasoning schedules and inefficient memory management. To overcome these limitations, we propose TAMP-Nav, a unified framework for efficient embodied navigation. First, we introduce a Pixel-to-3D Action Formulation (Point) that reformulates navigation into 2D visual prompting. Specifically, the VLM merely selects 2D pixels, which are then projected into 3D coordinates for a low-level SLAM controller. This design naturally aligns embodied execution with the VLM's inherent 2D visual capabilities. Second, we propose an integrated Selective Reasoning and Anchor-Trajectory Memory mechanism (Think and Memorize), which dynamically triggers Chain-of-Thought and retains high-fidelity memory only at critical nodes, compressing redundant trajectories into lightweight Space-Time Indicators, thereby preserving critical historical information and enhancing spatio-temporal perception. Finally, we design an efficient Two-Level Alignment Paradigm (Align) via Group Relative Policy Optimization (GRPO). By superimposing global outcome rewards with fine-grained process rewards, this dense supervision tightly aligns the agent's cognitive planning with physical environmental feedback, endowing the model with adaptive reasoning capabilities. Experiments demonstrate that TAMP-Nav achieves state-of-the-art performance (e.g., 66.2% SR on R2R-CE) with high runtime and sample efficiency (requiring only 90k training trajectories).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。