让机器人同时走路和精准操控末端,实现全身协同运动。
Multi-critic Learning for Whole-body End-effector Twist Tracking
- 采用多评论家架构分离行走与操作奖励,化解任务冲突。
- 基于速度的末端轨迹控制,实现平滑运动追踪。
- 硬件验证成功,机器人自动协调基座与手臂扩展工作空间。
单个策略同时学习步行与手臂运动面临挑战,因两者目标冲突:高效行走通常需要水平基座,而末端执行器追踪可能受益于基座倾斜以扩大可达范围。现有基于位姿的任务设定方法缺乏对末端速度的直接控制,难以实现平滑轨迹。为此,我们提出一种基于强化学习的框架,实现动态、速度感知的全身末端执行器控制。方法引入多评论家演员架构,解耦行走与操作的奖励信号,简化调参并更有效地处理任务冲突。同时设计基于旋量(twist)的末端任务形式,可追踪离散位姿与运动轨迹。通过四足机器人搭载机械臂的仿真与硬件实验验证,控制器能同步完成行走与末端执行器运动,展现出自发的全身协同行为,如基座辅助手臂扩展工作空间,尽管未显式建模该机制。视频与补充材料见 multi-critic-locomanipulation.github.io。
原文摘要 · Abstract (English)
Learning whole-body control for locomotion and arm motions in a single policy has challenges, as the two tasks have conflicting goals. For instance, efficient locomotion typically favors a horizontal base orientation, while end-effector tracking may benefit from base tilting to extend reachability. Additionally, current Reinforcement Learning (RL) approaches using a pose-based task specification lack the ability to directly control the end-effector velocity, making smoothly executing trajectories very challenging. To address these limitations, we propose an RL-based framework that allows for dynamic, velocity-aware whole-body end-effector control. Our method introduces a multi-critic actor architecture that decouples the reward signals for locomotion and manipulation, simplifying reward tuning and allowing the policy to resolve task conflicts more effectively. Furthermore, we design a twist-based end-effector task formulation that can track both discrete poses and motion trajectories. We validate our approach through a set of simulation and hardware experiments using a quadruped robot equipped with a robotic arm. The resulting controller can simultaneously walk and move its end-effector and shows emergent whole-body behaviors, where the base assists the arm in extending the workspace, despite a lack of explicit formulations. Videos and supplementary material can be found at multi-critic-locomanipulation.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。