arXiv:2508.16654cs.CV2025-08被引 9

提出MSNav框架,让机器人更智能地理解语言导航指令。

MSNav: Zero-Shot Vision-and-Language Navigation with Dynamic Memory and LLM Spatial Reasoning

  • 分模块设计:动态记忆、空间推理与大模型决策协同工作
  • 在R2R和REVERIE数据集上,成功率(SR)和路径加权成功率(SPL)显著提升
  • 专为长程任务优化,解决大模型记忆过载与空间推理弱的问题

视觉-语言导航(VLN)要求智能体理解自然语言指令并导航至复杂环境中的目标。现有方法多采用单一大型语言模型(LLM)端到端决策,存在空间推理能力差、跨模态对齐弱及长任务记忆过载等缺陷。为此,我们提出MSNav框架,融合三个模块:动态地图记忆模块通过选择性节点剪枝缓解记忆过载,支持长程探索;空间推理模块用于物体关系推断,提升终点识别准确率;决策模块基于LLM路径规划执行鲁棒动作。为支撑空间推理,我们构建了I-O-S数据集,并微调Qwen3-4B得到Qwen-Spatial(Qwen-Sp),其在物体列表提取任务中表现优于主流商用LLM,F1与NDCG得分更高。在Room-to-Room(R2R)和REVERIE数据集上的大量实验表明,MSNav达到当前最优性能,显著提升成功率为(SR)与路径加权成功率(SPL)。

原文摘要 · Abstract (English)

Vision-and-Language Navigation (VLN) requires an agent to interpret natural language instructions and navigate complex environments. Current approaches often adopt a "black-box" paradigm, where a single Large Language Model (LLM) makes end-to-end decisions. However, it is plagued by critical vulnerabilities, including poor spatial reasoning, weak cross-modal grounding, and memory overload in long-horizon tasks. To systematically address these issues, we propose Memory Spatial Navigation(MSNav), a framework that fuses three modules into a synergistic architecture, which transforms fragile inference into a robust, integrated intelligence. MSNav integrates three modules: Memory Module, a dynamic map memory module that tackles memory overload through selective node pruning, enhancing long-range exploration; Spatial Module, a module for spatial reasoning and object relationship inference that improves endpoint recognition; and Decision Module, a module using LLM-based path planning to execute robust actions. Powering Spatial Module, we also introduce an Instruction-Object-Space (I-O-S) dataset and fine-tune the Qwen3-4B model into Qwen-Spatial (Qwen-Sp), which outperforms leading commercial LLMs in object list extraction, achieving higher F1 and NDCG scores on the I-O-S test set. Extensive experiments on the Room-to-Room (R2R) and REVERIE datasets demonstrate MSNav's state-of-the-art performance with significant improvements in Success Rate (SR) and Success weighted by Path Length (SPL).

视觉语言导航大模型空间推理动态记忆

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。