arXiv:2511.17097cs.RO2025-11被引 5

让AI理解导航进展的语义层次,提升长程视觉语言导航准确性。

Progress-Think: Semantic Progress Reasoning for Vision-Language Navigation

  • 通过视觉历史与指令前缀的可微对齐,自动生成进展推理能力。
  • 在R2R-CE和RxR-CE上达到最优成功率与效率,优于现有方法。
  • 适合研究长程视觉语言导航、进展建模与强化学习融合的学者。

视觉语言导航要求智能体在长程任务中协同理解局部视觉上下文与任务进展程度。然而,现有视觉语言动作模型多聚焦直接动作预测,早期进展方法则仅预测数值进度;两者均忽视了观测序列与指令序列之间的单调共进特性。为此,Progress-Think提出语义进展推理机制,从视觉观测中预测指令风格的进展状态,实现更精准导航。为避免昂贵标注,我们设计三阶段框架:首阶段通过视觉历史与指令前缀的可微对齐,完成自对齐进展预训练;第二阶段将学习到的进展状态注入导航上下文,引导策略生成一致动作;第三阶段采用专门的进展感知强化学习目标,联合优化两个模块。在R2R-CE与RxR-CE数据集上的实验表明,该方法达到当前最优的成功率与效率,证明语义进展能提供更一致的导航进展表征。

原文摘要 · Abstract (English)

Vision-Language Navigation requires agents to act coherently over long horizons by understanding not only local visual context but also how far they have advanced within a multi-step instruction. However, recent Vision-Language-Action models focus on direct action prediction and earlier progress methods predict numeric achievements; both overlook the monotonic co-progression property of the observation and instruction sequences. Building on this insight, Progress-Think introduces semantic progress reasoning, predicting instruction-style progress from visual observations to enable more accurate navigation. To achieve this without expensive annotations, we propose a three-stage framework. In the initial stage, Self-Aligned Progress Pretraining bootstraps a reasoning module via a novel differentiable alignment between visual history and instruction prefixes. Then, Progress-Guided Policy Pretraining injects learned progress states into the navigation context, guiding the policy toward consistent actions. Finally, Progress-Policy Co-Finetuning jointly optimizes both modules with tailored progress-aware reinforcement objectives. Experiments on R2R-CE and RxR-CE show state-of-the-art success and efficiency, demonstrating that semantic progress yields a more consistent representation of navigation advancement.

视觉语言导航进展推理强化学习多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。