让机器人通过分步目标感知进度,自动纠错并恢复操作。
See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation
- 将语言指令分解为可执行的2D子目标,动态规划动作路径。
- 在LIBERO基准上比基线提升5%,在复杂任务中表现最优。
- 无需额外训练数据,适合真实场景中需容错的机器人操作。
通过显式、可行动的阶段性进展测量,实现鲁棒的机器人操作至关重要。这种进展感知使模型能定位当前任务状态,预测可验证的中间状态,并在进展停滞时检测与恢复失败。为此,我们提出见、规划、回溯(SPR)框架,该框架将语言指令动态转化为一系列空间子目标。SPR采用连续闭环流程:观察当前状态与下一个里程碑,规划至下一2D路点的轨迹,并在监测到进展偏离时,回溯至可恢复状态。该闭环机制实现鲁棒纠错,无需额外训练数据或辅助模型。大量实验表明,该框架在有效性、泛化性和鲁棒性方面表现优异:在LIBERO基准上比MolmoAct基线高出5%;在包含未见指令与初始状态的LIBERO-Plus挑战性基准上,表现出最先进的鲁棒性,性能下降最小,优于OpenVLA-OFT和UniVLA,展现出卓越的分布外鲁棒性。
原文摘要 · Abstract (English)
Measurement of task progress through explicit, actionable milestones is critical for robust robotic manipulation. This progress awareness enables a model to ground its current task status, anticipate verifiable intermediate states, and detect and recover from failures when progress stalls. To embody this capability, we introduce \textbf{S}ee, \textbf{P}lan, \textbf{R}ewind (SPR), a progress-aware vision-language-action framework that dynamically grounds language instructions into a sequence of spatial subgoals. SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure by monitoring progress against the expected sequence. This closed-loop approach enables robust error correction without requiring additional training data or auxiliary models. Extensive experiments demonstrate the framework's effectiveness, generalization and robustness: SPR outperforms the MolmoAct baseline by 5\% on the LIBERO benchmark. On the challenging LIBERO-Plus benchmark with unseen instructions and initial states, SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA, demonstrating superior out-of-distribution robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。