让AI导航更像人:用隐式推理实现快速精准路径规划
FantasyVLN: Unified Multimodal Chain-of-Thought Reasoning for Vision-Language Navigation
- 用压缩视觉潜空间替代显式图像生成,避免 token 膨胀
- 在 LH-VLN 数据集上成功率提升,推理延迟降低十倍
- 适合需要实时导航的机器人、自动驾驶场景
实现人类级视觉语言导航(VLN)需让智能体联合理解多模态指令与视觉空间上下文,并对长动作序列进行推理。近期工作如 NavCoT 和 NavGPT-2 表明链式思维(CoT)推理可提升可解释性与长程规划能力。多模态扩展如 OctoNav-R1 与 CoT-VLA 进一步验证了 CoT 作为类人导航推理的可行路径。但现有方法存在关键缺陷:纯文本 CoT 缺乏空间定位,易过拟合稀疏标注的推理步骤;多模态 CoT 则因生成想象中的视觉观测导致严重 token 膨胀,难以实现实时导航。本文提出 FantasyVLN,一种统一的隐式推理框架,在不增加显式 token 开销的前提下保留 CoT 推理优势。具体地,在 CoT 训练中,想象的视觉令牌通过预训练视觉自回归模型(VAR)编码至紧凑潜空间,模型在统一多 CoT 策略下联合学习文本、视觉与多模态 CoT 模式。推理时,模型直接完成指令到动作映射,同时保持推理感知表示。在 LH-VLN 上的大量实验表明,该方法实现了具有推理意识的实时导航,成功率达更高,效率显著提升,推理延迟相比显式 CoT 方法降低一个数量级。
原文摘要 · Abstract (English)
Achieving human-level performance in Vision-and-Language Navigation (VLN) requires an embodied agent to jointly understand multimodal instructions and visual-spatial context while reasoning over long action sequences. Recent works, such as NavCoT and NavGPT-2, demonstrate the potential of Chain-of-Thought (CoT) reasoning for improving interpretability and long-horizon planning. Moreover, multimodal extensions like OctoNav-R1 and CoT-VLA further validate CoT as a promising pathway toward human-like navigation reasoning. However, existing approaches face critical drawbacks: purely textual CoTs lack spatial grounding and easily overfit to sparse annotated reasoning steps, while multimodal CoTs incur severe token inflation by generating imagined visual observations, making real-time navigation impractical. In this work, we propose FantasyVLN, a unified implicit reasoning framework that preserves the benefits of CoT reasoning without explicit token overhead. Specifically, imagined visual tokens are encoded into a compact latent space using a pretrained Visual AutoRegressor (VAR) during CoT reasoning training, and the model jointly learns from textual, visual, and multimodal CoT modes under a unified multi-CoT strategy. At inference, our model performs direct instruction-to-action mapping while still enjoying reasoning-aware representations. Extensive experiments on LH-VLN show that our approach achieves reasoning-aware yet real-time navigation, improving success rates and efficiency while reducing inference latency by an order of magnitude compared to explicit CoT methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。