让机器人理解操作过程中的进展,提升长时序抓取任务的反馈精度。
ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulation

- 基于步骤结构与视觉变化,推理剩余动作再估算进度。
- 在6000万帧数据上训练,进度估计比基线更精细。
- 适合需要密集反馈的复杂机器人操作任务研究者。
长时序机器人操作需要反映任务在各阶段推进情况的密集反馈,而非仅关注最终结果。现有奖励模型常依赖轨迹级成功标签或时间插值,易将时间流逝误认为任务进展,无法捕捉未完成步骤、停滞或失败状态。本文提出ProcVLM,一种基于程序结构的视觉语言模型,可生成稠密的进度奖励信号。该模型不依赖终态结果或时间代理,而是将进度估计建立在程序结构和阶段内视觉变化基础上,并采用先推理后估计的范式,先预测剩余原子动作,再评估任务进展。我们通过合成帧级子任务语义标注,根据子任务结构分配进度预算,并基于子任务内视觉变化分配预算来构建监督信号。为规模化训练,我们构建了标准化的程序监督合成流水线,从30个具身数据集构建了包含6000万标注帧的ProcCorpus-60M,并从中提取ProcVQA用于程序感知预训练,以进度估计算为核心任务,辅以动作分割和未来规划。在ProcVQA及奖励模型基准测试中,ProcVLM显著提升了具身程序推理能力,且在轨迹内部进度估计上更具区分度,证明其作为下游奖励引导策略优化的稠密奖励模型的有效性。
原文摘要 · Abstract (English)
Long-horizon robotic manipulation requires dense feedback that reflects how a task advances through its procedural stages, not merely whether the final outcome is successful. Existing reward models often rely on trajectory-level success labels or time-based interpolation, which can conflate elapsed time with true task progress and therefore fail to capture unfinished steps, stagnation, and failure states. We present ProcVLM, a progress-aware vision-language model that learns procedure-grounded progress as a dense reward signal for manipulation. Rather than deriving progress from terminal outcomes or temporal proxies, ProcVLM grounds progress estimation in procedural structure and intra-stage visual change, and further adopts a reasoning-before-estimation paradigm that infers the remaining atomic actions before estimating task progress. Specifically, we construct this supervision by synthesizing frame-level subtask-semantic annotations, assigning progress budgets according to subtask structure, and distributing each budget based on intra-subtask visual change. To train ProcVLM at scale, we build a standardized procedural supervision synthesis pipeline and construct ProcCorpus-60M from 30 embodied datasets with 60M annotated frames, from which we derive ProcVQA for procedure-aware pretraining, with progress estimation as the central task alongside action segmentation and future planning. Experiments on ProcVQA and reward-model benchmarks show that ProcVLM improves embodied procedural reasoning and yields more discriminative trajectory-internal progress estimates than representative baselines, supporting its use as a dense reward model for downstream reward-guided policy optimization. Project page: https://procvlm.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。