arXiv:2608.03753cs.RO2026-08

用物体图结构自动分解长程操作任务,让机器人学会分步完成复杂动作。

GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation

论文配图:GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation
图 1 · 摘自论文原文
  • 用物体和空间关系构建场景图,通过自监督学习提取任务相关特征。
  • 基于隐空间距离生成密集奖励,使长任务自动分解为可执行子任务。
  • 在7个复杂任务上实现74.4%成功率,显著优于基线和人工标注方案。

长程操作技能的强化学习仍面临奖励设计复杂、稀疏奖励引导不足及人工标注成本高的挑战。视觉示范可提供奖励学习监督,但基于原始像素的奖励易受视觉变化、背景干扰和机器人运动影响。本文提出GORDON,一种基于图的物体中心奖励学习框架,从无动作视频示范中学习密集奖励。每个视觉场景被表示为检测物体及其空间关系构成的图,使用图神经网络在自监督下将这些图嵌入任务对齐的隐空间。为对齐语义任务进展,引入活动感知加权池化机制,突出任务相关物体并抑制机器人主导的运动。密集奖励通过当前状态与示范目标配置在学习隐空间中的距离计算,衡量任务进展。在长程任务中,该奖励的时间轨迹揭示阶段性的物体状态转移,实现无需人工分割的自动子任务发现。所发现的段落用于训练子任务特定奖励与专用策略,并顺序组合。在MAGICAL和ManiSkill3基准上的七个操纵任务实验表明,该物体中心奖励在短程设置中提升强化学习性能,并通过自动分解成功实现复杂长程任务的策略学习,长程任务平均成功率达74.4%,较最优基线提升约35个百分点,较人工黄金标准提升约25个百分点。

原文摘要 · Abstract (English)

Learning long-horizon manipulation skills with reinforcement learning remains challenging due to the complexity of reward design, the limited guidance of sparse rewards, and the high cost of manual subtask annotation. Visual demonstrations can provide supervision for reward learning, but rewards learned from raw pixels can be brittle and sensitive to visual variation, background appearance, and robot motion. In this work, we propose GORDON, a graph-based object-centric reward learning framework that learns dense rewards from action-free video demonstrations. Each visual scene is represented as a graph of detected objects and spatial relations, and a graph neural network is trained in a self-supervised manner to embed these graphs into a task-aligned latent space. To align the representation with semantic task progress, we introduce an activity-aware weighted pooling mechanism that emphasizes task-relevant objects while masking robot-dominated motion. The dense reward is then computed as distances in the learned latent space of the current state to demonstrated goal configurations, providing a measure of task progress. In long-horizon tasks, the temporal profile of this reward reveals stage-wise object-state transitions, enabling automatic subtask discovery without manual segmentation. The discovered segments are then used to train subtask-specific rewards and specialized policies that are composed sequentially. Experiments on seven manipulation tasks on MAGICAL and ManiSkill3 benchmarks show that our object-centric reward improves reinforcement learning in short-horizon settings and enables successful policy learning in complex long-horizon tasks through automatic decomposition, achieving an average success rate of 74.4% across the long-horizon tasks (on average approximately +35 p.p. vs. best learned baseline and approximately +25 p.p. vs. oracle).

强化学习任务分解物体中心长程操作

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。