arXiv:2601.12766cs.CVcs.SY2026-01被引 5

用空间感知提升语言导航,让智能体在复杂环境更准地听懂指令。

Spatial-VLN: Zero-Shot Vision-and-Language Navigation With Explicit Spatial Perception and Exploration

  • 通过全景过滤与专家模块增强空间感知,实现跨视角一致的视觉理解。
  • 在VLN-CE上用低成本LLM达到顶尖表现,真实场景测试验证强泛化能力。
  • 主动探查机制解决模糊指令问题,适合复杂室内导航任务。

零样本视觉-语言导航(VLN)代理虽具备良好泛化能力,但在复杂连续环境中仍受限于空间感知不足。针对门交互、多房间导航和指令歧义三大关键挑战,本文提出空间感知引导的探索框架Spatial-VLN。该框架包含两个核心模块:空间感知增强(SPE)模块融合全景过滤与专用门及区域专家,生成具空间一致性与跨视图一致性的感知表征;在此基础上,探索式多专家推理(EMR)模块采用并行大语言模型专家处理路径点语义与区域间空间转换。当专家预测出现分歧时,触发查询-探索机制,主动探测关键区域以消除感知歧义。在VLN-CE上的实验表明,Spatial-VLN仅使用低成本大语言模型即达到当前最优性能。为进一步验证真实世界适用性,引入基于价值的路径点采样策略,有效弥合仿真到现实的差距。大量真实场景评估证实,该框架在复杂环境中具有卓越的泛化与鲁棒性。代码与视频见https://yueluhhxx.github.io/Spatial-VLN-web/。

原文摘要 · Abstract (English)

Zero-shot Vision-and-Language Navigation (VLN) agents leveraging Large Language Models (LLMs) excel in generalization but suffer from insufficient spatial perception. Focusing on complex continuous environments, we categorize key perceptual bottlenecks into three spatial challenges: door interaction,multi-room navigation, and ambiguous instruction execution, where existing methods consistently suffer high failure rates. We present Spatial-VLN, a perception-guided exploration framework designed to overcome these challenges. The framework consists of two main modules. The Spatial Perception Enhancement (SPE) module integrates panoramic filtering with specialized door and region experts to produce spatially coherent, cross-view consistent perceptual representations. Building on this foundation, our Explored Multi-expert Reasoning (EMR) module uses parallel LLM experts to address waypoint-level semantics and region-level spatial transitions. When discrepancies arise between expert predictions, a query-and-explore mechanism is activated, prompting the agent to actively probe critical areas and resolve perceptual ambiguities. Experiments on VLN-CE demonstrate that Spatial VLN achieves state-of-the-art performance using only low-cost LLMs. Furthermore, to validate real-world applicability, we introduce a value-based waypoint sampling strategy that effectively bridges the Sim2Real gap. Extensive real-world evaluations confirm that our framework delivers superior generalization and robustness in complex environments. Our codes and videos are available at https://yueluhhxx.github.io/Spatial-VLN-web/.

视觉语言导航空间感知大语言模型智能体探索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。