用记忆机制让导航模型又快又准，实时运行达14帧每秒。
MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
- 分层记忆管理：短期感知精细处理，长期记忆压缩存储。
- 动作提速：用原子级中间动作替代逐词生成，提升推理速度7倍。
- 适合做智能导航的模型开发，尤其关注实时性与长程一致性。
视觉-语言导航在连续环境（VLN-CE）中要求智能体保持长期视觉历史以保证轨迹一致性，同时执行动作需低延迟。现有基于视频的方法难以兼顾两者。我们提出MemVLN框架，实现当前最佳性能且支持实时推理（14 FPS）。该框架使用视觉编码器处理连续观测，结合大语言模型理解指令并生成动作。核心创新在于分层情景记忆机制：对近期感知精细处理，长期记忆则压缩保存。此外，引入程序化记忆，采用紧凑的原子级中层动作词汇表，跳过自回归解码延迟。在VLN-CE数据集上，MemVLN-4B在R2R任务中比基线Qwen3-VL-4B提升5.8%成功率，在RxR任务中提升9.7%，推理延迟降低7倍。
原文摘要 · Abstract (English)
Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires agents to maintain long-horizon visual history for trajectory consistency while executing actions with low latency. Existing video-based VLN approaches typically struggle to satisfy both demands simultaneously. To address these challenges, we propose MemVLN, a novel VLN framework that achieves state-of-the-art performance with real-time inference efficiency (14 FPS). MemVLN utilizes a visual encoder to process continuous observations and a Large Language Model (LLM) to interpret instructions and generate actions. Central to our approach is an Episodic Memory management that applies pyramidal resolutions. This mechanism concentrates computation on immediate percepts while retaining compressed long-term history. Complementing to this design, we introduce Procedural Memory for fast action with a compact vocabulary of atomic mid-level actions to bypass auto-regressive decoding latency. Experiments on VLN-CE show that MemVLN-4B surpasses the baseline Qwen3-VL-4B architecture by 5.8\% SR in R2R and 9.7\% SR in RxR, while achieving a 7$\times$ speedup in inference latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。