通过对称性并行化加速机器人端强化学习，实现分钟级训练。
SymmGrid: Super-Scaling On-Robot Learning with Parallelized Symmetries and Egocentric-Exocentric Visual Perception
- 基于对称树结构生成并行化状态变换，构建几何网格式经验池。
- 实测训练速度提升1.37-2.17倍，成功率达1.09-1.27倍，最快仅需79.3分钟。
- 适用于机械臂与人形机器人，特别适合视觉与本体感知融合任务。
直接在物理机器人上进行深度强化学习（即机器人端学习）仍受限于缓慢的墙钟训练时间。本文提出SymmGrid，一种受并行对称性启发的轨迹级增强框架，可显著加速在自我中心与外部视角视觉设置下的机器人学习。我们基于对称树建模马尔可夫决策过程，使状态-动作对具备可并行化的不变变换，形成几何网格结构。状态由自摄或外摄图像及本体感知信息构成，后者需通过单应性变换以匹配空间变换。这些并行变换生成大量唯一对称等价物，丰富且一致地填充重放缓冲区，加快学习并提升性能。我们在真实机器人操作任务中进行了广泛训练与评估，包括插销、电缆布线和物体重定位。相比最先进方法，SymmGrid实现1.37-2.17倍的墙钟训练加速，评估成功率提升1.09-1.27倍，最快训练收敛时间分别为16.6、10.9和79.3分钟。轨迹级评估采用归一化曲线下面积（nAUC）比值，最高提升达2.59倍。结果表明，简单分支对称性经超规模放大后可产生显著效果，推动实现机械臂与人形机器人操纵任务的亚10分钟机器人端学习。项目页面见 symmgrid-robot.github.io。
原文摘要 · Abstract (English)
Deep reinforcement policy learning directly in physical robots (on-robot learning) remains bottlenecked by slow wall-clock training times. We present SymmGrid, a trajectory level augmentation framework inspired by parallelized symmetries that super-scales group transformations to significantly accelerate on-robot learning in both egocentric and exocentric visual setups. We model a Markov Decision Process (MDP) under a symmetry tree, in which state-action pairs have admissible parallelized invariant transformations that yield a geometric grid structure. The state is modelled with ego- or exocentric images and proprioception information. The latter require special treatment, in the form of homographies, to warp visual scenes in line with their corresponding spatial transformations. These parallelized transformations produce a large set of unique symmetric equivalences that populate the replay buffer with diverse and consistent experiences that speed up learning and improve performance. We present extensive training and evaluations performed directly on real robot manipulation contact tasks including peg-insertions, cable routing, and object relocations. Relative to SOTA, SymmGrid achieved wall-clock training convergence speed-ups of 1.37-2.17x, evaluation success rate improvements of 1.09x-1.27x, fastest training convergence times of 16.6, 10.9, and 79.3 minutes respectively. For trajectory wide assessments, we used normalized area under the curve (nAUC) ratios. SymmGrid achieved improvements of up to 2.59x. These results confirm that simple branch symmetries can have an outsized result due to super-scaling and bring us closer to sub-10 minute on-robot learning training in manipulation tasks suitable for arms and humanoids. The project page is available at symmgrid-robot.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。