让机器人从自然语言指令中生成可验证的长程操作计划,提升真实场景下的任务成功率。
Reinforced Embodied Planning with Verifiable Reward for Real-World Robotic Manipulation
- 用视觉语言模型生成带思维链的连贯操作计划,确保物理合理性和逻辑一致性。
- 通过结构化奖励机制评估计划与真实动作序列的匹配度,提升规划准确性。
- 在真实机器人上实现60%的任务成功率提升,适合具身智能与自主机器人研究者。
让机器人从自由形式的语言指令执行长程操作任务仍是具身人工智能中的根本挑战。尽管视觉语言模型(VLM)在高层规划中展现出潜力,但其在真实世界中的部署受限于两大问题:(i) 缺乏大规模、顺序性操作数据,无法将自然语言与多步动作计划关联;(ii) 缺少密集且可解释的奖励信号用于微调VLM以优化规划目标。为此,我们提出REVER框架,使VLM能在真实场景中根据自然语言指令生成并验证长程操作计划。在此框架下,我们训练并发布RoboFarseer,一个激励生成思维链的VLM,具备时空推理能力,确保计划在物理上可行且逻辑连贯。为获取训练数据,我们利用通用操作接口框架捕获硬件无关的原子技能演示,并通过自动化标注引擎将其转换为视觉-指令-计划三元组。我们引入可验证奖励,通过生成计划与真实技能序列之间的有序二分图匹配重叠度进行评分。运行时,微调后的VLM兼具规划与逐步监控功能。RoboFarseer性能媲美甚至超过规模大数个数量级的专有模型,在开放式规划任务中优于最佳基线超40%。在真实世界长程任务中,该系统相较无规划器的底层控制器整体成功率提升约60%。论文发表后将开源数据集与训练模型。
原文摘要 · Abstract (English)
Enabling robots to execute long-horizon manipulation tasks from free-form language instructions remains a fundamental challenge in embodied AI. While vision-language models (VLMs) have shown promise as high-level planners, their deployment in the real world is hindered by two gaps: (i) the scarcity of large-scale, sequential manipulation data that couples natural language with multi-step action plans, and (ii) the absence of dense, interpretable rewards for fine-tuning VLMs on planning objectives. To address these issues, we propose REVER, a framework that empowers VLMs to generate and validate long-horizon manipulation plans from natural language instructions in real-world scenarios. Under REVER we train and release RoboFarseer, a VLM incentivized to emit chain-of-thought that perform temporal and spatial reasoning, ensuring physically plausible and logically coherent plans. To obtain training data, we leverage the Universal Manipulation Interface framework to capture hardware-agnostic demonstrations of atomic skills. An automated annotation engine converts each demonstration into vision-instruction-plan triplet. We introduce a verifiable reward that scores the generated plan by its ordered bipartite matching overlap with the ground-truth skill sequence. At run time, the fine-tuned VLM functions both as a planner and as a monitor, verifying step-wise completion. RoboFarseer matches or exceeds the performance of proprietary models that are orders of magnitude larger, while on open-ended planning it surpasses the best baseline by more than 40%. In real-world, long-horizon tasks, the complete system boosts overall success by roughly 60% compared with the same low-level controller without the planner. We will open-source both the dataset and the trained model upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。