让视觉语言导航模型像对话一样多轮决策,提升长程任务成功率。
LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigation
- 设计多轮强化学习框架,模拟智能体与环境的连续交互对话。
- 在4000条轨迹上使成功率从64.3%提升至73.0%,显著优于现有方法。
- 适合追求长程导航泛化能力与抗退化性能的研究者。
本文提出LongNav-R1,一种面向长程视觉-语言-动作(VLA)导航的端到端多轮强化学习框架。不同于传统单轮范式,LongNav-R1将导航决策过程重构为智能体与具身环境间的连续多轮对话。该框架具备双重优势:一是使智能体能推理历史交互的因果效应与未来序列结果;二是支持直接从在线交互中学习,促进轨迹多样性,避免人类示范带来的行为僵化。此外,引入视野自适应策略优化机制,显式处理优势估计中的变长时域问题,实现长序列上的精准时序信用分配。实验表明,在物体导航基准测试中,使用4000条采样轨迹,LongNav-R1将Qwen3-VL-2B的成功率从64.3%提升至73.0%,展现优异样本效率,并显著超越当前最优方法。模型在真实世界长程导航场景中实现零样本表现,验证其泛化性与鲁棒性。所有源代码将在发表后开源。
原文摘要 · Abstract (English)
This paper develops LongNav-R1, an end-to-end multi-turn reinforcement learning (RL) framework designed to optimize Visual-Language-Action (VLA) models for long-horizon navigation. Unlike existing single-turn paradigm, LongNav-R1 reformulates the navigation decision process as a continuous multi-turn conversation between the VLA policy and the embodied environment. This multi-turn RL framework offers two distinct advantages: i) it enables the agent to reason about the causal effects of historical interactions and sequential future outcomes; and ii) it allows the model to learn directly from online interactions, fostering diverse trajectory generation and avoiding the behavioral rigidity often imposed by human demonstrations. Furthermore, we introduce Horizon-Adaptive Policy Optimization. This mechanism explicitly accounts for varying horizon lengths during advantage estimation, facilitating accurate temporal credit assignment over extended sequences. Consequently, the agent develops diverse navigation behaviors and resists collapse during long-horizon tasks. Experiments on object navigation benchmarks validate the framework's efficacy: With 4,000 rollout trajectories, LongNav-R1 boosts the Qwen3-VL-2B success rate from 64.3% to 73.0%. These results demonstrate superior sample efficiency and significantly outperform state-of-the-art methods. The model's generalizability and robustness are further validated by its zero-shot performance in long-horizon real-world navigation settings. All source code will be open-sourced upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。