arXiv:2604.27472cs.AIcs.LG2026-04被引 3

用对比强化学习让机器人理解任务进展,提升复杂指令执行成功率。

PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations

论文配图:PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations
图 1 · 摘自论文原文
  • 将语言指令视为目标,通过对比学习构建状态-动作与目标的统一嵌入空间。
  • 在无奖励标注的离线轨迹上训练,实现1670亿词令牌预训练,性能领先。
  • 特别适合长时序、需接触的复杂任务,零样本新指令泛化能力更强。

视觉-语言-动作(VLA)模型通过强大的视觉-语言先验推动机器人控制发展。然而,现有VLA主要将预训练视为监督行为克隆,忽略了机器人学习本质上是目标达成过程,需理解时间维度的任务进展。本文提出PRTS(Primitive Reasoning and Tasking System),一种通过目标条件强化学习重构预训练的VLA基础模型。将语言指令视为目标,采用对比强化学习,使状态-动作嵌入与目标嵌入的内积近似对数折扣目标可达性,即从当前状态-动作到达指定语言目标的概率,定量评估物理可行性,超越静态语义匹配。该模型直接从离线轨迹中提取密集的目标可达性监督,无需奖励标注,并通过角色感知因果掩码融入VLM主干,开销极低。这一范式赋予高层推理系统内在的目标可达性意识,连接语义推理与时间任务进展,进一步提升目标条件动作预测能力。在1670亿词令牌的多样化操作与具身推理数据上预训练后,PRTS在LIBERO、LIBERO-Pro、LIBERO-Plus、SimplerEnv及包含14个复杂任务的真实世界套件上均达最先进水平,尤其在长时序、高接触和零样本新指令场景下表现显著提升,验证了注入目标可达性意识能显著改善通用机器人基础策略的执行成功率与长时序规划能力。

原文摘要 · Abstract (English)

Vision-Language-Action (VLA) models advance robotic control via strong visual-linguistic priors. However, existing VLAs predominantly frame pretraining as supervised behavior cloning, overlooking the fundamental nature of robot learning as a goal-reaching process that requires understanding temporal task progress. We present \textbf{PRTS} (\textbf{P}rimitive \textbf{R}easoning and \textbf{T}asking \textbf{S}ystem), a VLA foundation model that reformulates pretraining through Goal-Conditioned Reinforcement Learning. By treating language instructions as goals and employing contrastive reinforcement learning, PRTS learns a unified embedding space where the inner product of state-action and goal embeddings approximates the log-discounted goal occupancy, the probability of reaching the language-specified goal from the current state-action, quantitatively assessing physical feasibility beyond static semantic matching. PRTS draws this dense goal-reachability supervision directly from offline trajectories without reward annotations, and folds it into the VLM backbone via a role-aware causal mask, incurring negligible overhead over vanilla behavior cloning. This paradigm endows the high-level reasoning system with intrinsic goal reachability awareness, bridging semantic reasoning and temporal task progress, and further benefits goal-conditioned action prediction. Pretrained on 167B tokens of diverse manipulation and embodied-reasoning data, PRTS reaches state-of-the-art performance on LIBERO, LIBERO-Pro, LIBERO-Plus, SimplerEnv, and a real-world suite of 14 complex tasks, with particularly substantial gains on long-horizon, contact-rich, and zero-shot novel-instruction settings, confirming that injecting goal-reachability awareness significantly improves both execution success and long-horizon planning of general-purpose robotic foundation policies.

机器人控制强化学习多模态目标任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。