arXiv:2604.27620cs.CV2026-04被引 3

让视觉语言模型学会看懂动作因果与未来变化,提升导航能力。

SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigation

论文配图:SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigation
图 1 · 摘自论文原文
  • 通过反向推理动作和预测未来视觉变化,激活模型空间感知
  • 在VLN-CE上超越现有方法,长程导航准确率显著提升
  • 适合研究智能体导航、多模态学习的学者参考

视觉语言导航(VLN)旨在使具身智能体在未见过的3D环境中,根据自然语言指令导航至目标位置。本文认为,将视觉语言模型(VLM)适配于VLN需赋予其两种互补能力:后向动作推理(为何执行该动作)与前向状态转移预测(下一步如何变化)。基于此,提出SpaAct框架,通过两个空间激活任务实现:动作回溯(从视觉变化推断已执行动作序列),以及未来帧选择(基于历史与动作预测下一帧视觉变化)。这两个目标为后向推理与前向预测提供轻量监督,以适应性方式增强模型动态空间感知。为进一步稳定训练,设计三因子渐进式自适应课程学习(TriPA),按难易程度组织样本,使模型逐步掌握从基础移动到长时序推理的导航技能。在标准VLN-CE基准测试中,SpaAct持续提升基于VLM的导航性能,达到当前最优水平。代码与模型将公开,支持后续研究。

原文摘要 · Abstract (English)

Vision-and-Language Navigation (VLN) aims to enable an embodied agent to follow natural-language instructions and navigate to a target location in unseen 3D environments. We argue that adapting VLMs to VLN requires endowing them with two complementary capabilities for acquiring such awareness, namely backward action reasoning (why) and forward transition prediction~(how). Based on this insight, we propose SpaAct, a simple yet effective training framework that activates the dynamic spatial awareness in VLMs. Specifically, SpaAct introduces two spatial activation tasks: Action Retrospection, which asks the model to infer the executed action sequence from visual transitions, and Future Frame Selection, which forces the model to predict the visual transitions conditioned on history and action. These two objectives provide lightweight supervision on both backward action reasoning and forward transition prediction, encouraging the model to build dynamic spatial awareness in a VLM-friendly way. To further stabilize adaptation, we design TriPA, a Tri-factor Progressive Adaptive curriculum learning method that organizes training samples from easy to hard, allowing the model to gradually acquire navigation skills from basic locomotion to long-horizon reasoning. Experiments on standard VLN-CE benchmarks show that SpaAct consistently improves VLM-based navigation and achieves state-of-the-art performance. We will release the code and models to support future research.

视觉导航多模态学习自适应训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。