用大模型直接从视频流生成连续导航动作,提升智能体行走能力。
VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning
- 基于大视觉语言模型,直接将第一视角视频转为连续动作
- 在VLN-CE上达到新高,成功率超基线15.2个百分点
- 适合做具身智能、机器人导航研究的开发者参考
视觉语言导航(VLN)是具身人工智能的核心挑战,要求智能体根据自然语言指令在真实环境中导航。当前基于语言模型的导航系统依赖离散拓扑图,路径规划受限于预设节点连接。本文提出端到端的VLN-R1框架,利用大视觉语言模型(LVLM)直接将第一人称视频流转换为连续导航动作,并采用受DeepSeek-R1启发的GRPO训练策略。为实现有效训练,我们首先在3D模拟器Habitat中构建VLN-Ego数据集,并提出长短时记忆采样以平衡历史与当前观测。尽管大语言模型可监督完整文本指令,但缺乏细粒度动作级控制。本框架采用两阶段训练:a)监督微调(SFT)对齐模型动作序列预测与专家示范;b)强化微调(RFT)结合时间衰减奖励(TDR)机制,战略性加权多步未来动作。实验表明,VLN-R1在VLN-CE基准上表现优异,证明了LVLM可通过数据高效、奖励驱动的后训练实现具身导航与任务特定推理。
原文摘要 · Abstract (English)
Vision-Language Navigation (VLN) is a core challenge in embodied AI, requiring agents to navigate real-world environments using natural language instructions. Current language model-based navigation systems operate on discrete topological graphs, limiting path planning to predefined node connections. We propose VLN-R1, an end-to-end framework that leverages Large Vision-Language Models (LVLM) to directly translate egocentric video streams into continuous navigation actions, adopting GRPO-based training inspired by DeepSeek-R1. To enable effective training, we first construct the VLN-Ego dataset using a 3D simulator, Habitat, and propose Long-Short Memory Sampling to balance historical and current observations. While large language models can supervise complete textual instructions, they lack fine-grained action-level control. Our framework employs a two-stage training approach: a) Supervised fine-tuning (SFT) to align the model's action sequence text predictions with expert demonstrations, followed by b) Reinforcement fine-tuning (RFT) enhanced with a Time-Decayed Reward (TDR) mechanism that strategically weights multi-step future actions. Experimental results show VLN-R1 achieves strong performance on VLN-CE benchmark. VLN-R1 proves LVLMs can drive embodied navigation and enhance task-specific reasoning through data-efficient, reward-driven post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。