arXiv:2606.16978cs.ROcs.LG2026-06被引 1

用任务误差指导学习,实现机械臂稳定接五球。

Task-Error Residual Learning for Real-Robot Five-Ball Juggling

论文配图:Task-Error Residual Learning for Real-Robot Five-Ball Juggling
图 1 · 摘自论文原文
  • 以任务误差方向作为反馈信号,提升样本效率
  • 从第二次尝试开始稳定接球,无需失败
  • 适合对真实机器人强化学习感兴趣的工程师

对于优化已有行为的残差学习,样本效率取决于每次试错返回的信息量以及学习者利用信息的效率。强化学习的标准标量奖励提供的信息远少于定义任务的方向性任务误差,而随机探索进一步丢弃了每次试错所携带的信息。通过方向性任务误差监督的残差学习和驱动样本选择的任务误差模型,本工作在拟人化的Barrett WAM机械臂上实现了稳定的三、四、五球抛接。尽管规划与控制基于一个简单理想化的堆栈,系统在第二次尝试后即收敛。首次尝试失败后,任务误差单调下降且不再出现失误。相比之下,人类通常需要数年练习才能完成五球抛接。我们在两个三元轴上对比残差学习器:学习反馈中的方向性信息与分析先验的确定性,涵盖牛顿型雅可比更新、复合贝叶斯优化与随机搜索方法。两个维度均至关重要:仅方向反馈或仅信息性先验均不足,而结合两者的最简方法——固定雅可比牛顿更新,最为可靠。学习到的残差能容忍显著的先验偏差和关节跟踪退化,主要影响收敛速度。因此,真实机器人上残差学习的瓶颈在于监督信号的信息量及其利用方式,而非周围堆栈的精度。所有实验视频见 https://kai-ploeger.com/residual-juggling。

原文摘要 · Abstract (English)

For residual learning that refines existing behavior, sample efficiency depends on two things: how much information each rollout returns, and how efficiently the learner uses that information. Reinforcement learning's standard scalar reward carries far less information than the directional task error that defines the task. Random exploration further discards whatever information each rollout returns. Through residual learning with directional task-error supervision and a task error model that drives sample selection, we achieve stable three-, four-, and five-ball juggling on anthropomorphic Barrett WAM arms. Despite planning and controlling through a simple, idealized stack, the system converges from the second attempt. The first attempt drops, after which task error decreases monotonically without further failures. In comparison, five-ball juggling typically takes humans years of practice. We compare residual learners across two ternary axes, the directional information in the learning feedback and the commitment of the analytic prior, spanning Newton-style Jacobian updates, Composite Bayesian Optimization, and stochastic search methods. Both axes prove necessary: neither directional feedback nor an informative prior suffices alone, and the simplest method that combines them, a fixed-Jacobian Newton update, is the most reliable. The learned residual tolerates substantial prior misalignment and degraded joint tracking, affecting mainly convergence speed. The bottleneck for residual learning on real robots is therefore the information content of the supervision signal and how the learner uses it, not the accuracy of the surrounding stack. Video documentation of all experiments is available at https://kai-ploeger.com/residual-juggling.

机器人控制残差学习强化学习五球抛接

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。