通过三步视觉导航策略提升零样本语言导航成功率
Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigation
- 分三步：看前方提取全局路标，看当下对齐子目标，回头看修正轨迹偏差
- 在R2R-CE和RxR-CE数据集上达到当前最优零样本表现
- 无需微调，可无缝接入现有导航系统，适合快速部署
基于多模态大语言模型（MLLM）的视觉导航在未知环境中取得了突破性进展。这些模型能在每个时间步评估当前视图与任务目标，规划运动序列。然而，现有的零样本视觉-语言导航（VLN）代理仍易偏离路径、提前停止，整体成功率达不到理想水平。本文提出三步导航（Three-Step Nav），采用三视协议：首先“向前看”以提取全局路标并绘制粗略路线；接着“此刻看”将当前视觉观测与下一子目标对齐，实现细粒度引导；最后“向后看”审计整个轨迹，纠正累积漂移后再终止。该规划器无需梯度更新或任务特定微调，可低开销嵌入现有VLN流程。在R2R-CE和RxR-CE数据集上，Three-Step Nav实现了当前最佳的零样本性能。代码已开源：https://github.com/ZoeyZheng0/3-step-Nav。
原文摘要 · Abstract (English)
Breakthrough progress in vision-based navigation through unknown environments has been achieved by using multimodal large language models (MLLMs). These models can plan a sequence of motions by evaluating the current view at each time step against the task and goal given to the agent. However, current zero-shot Vision-and-Language Navigation (VLN) agents powered by MLLMs still tend to drift off course, halt prematurely, and achieve low overall success rates. We propose Three-Step Nav to counteract these failures with a three-view protocol: First, "look forward" to extract global landmarks and sketch a coarse plan. Then, "look now" to align the current visual observation with the next sub-goal for fine-grained guidance. Finally, "look backward" audits the entire trajectory to correct accumulated drift before stopping. Requiring no gradient updates or task-specific fine-tuning, our planner drops into existing VLN pipelines with minimal overhead. Three-Step Nav achieves state-of-the-art zero-shot performance on the R2R-CE and RxR-CE dataset. Our code is available at https://github.com/ZoeyZheng0/3-step-Nav.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。