arXiv:2605.09410cs.ROcs.AI2026-05被引 1

让视觉语言动作模型学会从失败中恢复,提升长程操作鲁棒性。

RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models

论文配图:RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models
图 1 · 摘自论文原文
  • 区分成功、失败与恢复轨迹,用修复片段重置状态以纠正偏差。
  • 在模拟和真实双臂任务中,对抗性成功率从20%提升至75%,最高达80%。
  • 无需在线检测失败或手动重试,部署时固定高价值可自动导向成功路径。

视觉-语言-动作(VLA)模型在长程、接触密集型操作中仍易失效,因仅依赖成功样本的模仿学习难以提供执行漂移的足够监督,且失败轨迹常被丢弃。本文提出RePO-VLA,一种面向恢复的策略优化框架,为成功、恢复与失败轨迹分配不同角色。首先采用恢复感知初始化(RAI),切分恢复段并重置历史,使修正动作基于当前不利状态而非先前失败。随后学习一个进度感知语义价值函数(PAS-VF),将时空轨迹特征与指令及成功参考对齐。由此生成的标签通过可靠性衰减机制保留有价值的失败前缀,低值标签则标记漂移与终态崩溃,从而区分正常、失败与修正动作。数据引擎将不利状态转化为规划生成或人工收集的修正轨迹,训练模型回归成功流形。价值条件精炼(VCR)引导策略偏好高进展动作。部署时,固定高值(v=1.0)即可偏向学习到的成功流形,无需在线失败检测或启发式重试。引入FRBench基准,含标准化错误注入与恢复评估。在模拟与真实双臂任务中,RePO-VLA显著提升鲁棒性,平均对抗性成功率由20%提升至75%,真实场景下最高达80%。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models remain brittle in long-horizon, contact-rich manipulation because success-only imitation provides little supervision for execution drift, while failed rollouts are often discarded. We introduce RePO-VLA, a recovery-driven policy optimization framework that assigns distinct roles to success, recovery, and failure trajectories. RePO-VLA first applies Recovery-Aware Initialization (RAI), slicing recovery segments and resetting history so corrective actions depend on the current adverse state rather than the preceding failure. It then learns a Progress-Aware Semantic Value Function (PAS-VF), aligning spatiotemporal trajectory features with instructions and successful references. The resulting labels salvage useful failure prefixes via reliability decay, while low-value labels mark drift and terminal breakdowns, teaching differences among nominal, failed, and corrective actions. The data engine turns adverse states into planner-generated or human-collected corrective rollouts, teaching recovery to the success manifold. Value-Conditioned Refinement (VCR) trains the policy to prefer high-progress actions. At deployment, a fixed high value ($v=1.0$) biases actions toward the learned success manifold without online failure detectors or heuristic retries. We introduce FRBench, with standardized error injection and recovery-focused evaluation. Across simulated and real-world bimanual tasks, RePO-VLA improves robustness, raising adversarial success from 20% to 75% on average and up to 80% in scaled real-world trials.

强化学习机器人操作多模态恢复机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。