让机器人通过触觉感知人类意图,协同搬运物体。
H2-COMPACT: Human-Humanoid Co-Manipulation via Adaptive Contact Trajectory Policies
- 分层策略:触觉信号转为运动指令,再生成稳定步态。
- 实测表现媲美蒙眼人类跟随者,支持0-3公斤负载自适应。
- 无需标记物,仅靠视频与力传感器实现人机协同。
我们提出一种分层策略学习框架,使足式人形机器人仅通过双腕传感器的六维力/扭矩信号,即可与人类搭档协作搬运长条形负载。上层采用轻量级行为克隆网络,处理双臂传感器的力/扭矩流,输出全身平面速度指令以捕捉人类施力意图;下层采用深度强化学习策略,在Isaac Gym中针对0-3公斤随机负载和摩擦条件训练,并在MuJoCo及真实Unitree G1机器人上验证,将高层速度指令映射为稳定、承重下的关节轨迹。通过解耦意图解析(力→速度)与足式运动控制(速度→关节),方法兼具对人类输入的直观响应性与强负载自适应行走能力。训练数据仅依赖同步的RGB视频与力/扭矩读数,使用SAM2和WHAM提取三维人体姿态与速度,无需动作捕捉或标记。真实实验中,机器人在完成时间、轨迹偏差、速度同步性和跟从力等指标上达到与蒙眼人类跟随基线相当的协作搬运性能。本工作首次实现学习型触觉引导与全身足式控制融合,达成流畅的人机协同操作。代码与视频已公开于H2-COMPACT网站。
原文摘要 · Abstract (English)
We present a hierarchical policy-learning framework that enables a legged humanoid to cooperatively carry extended loads with a human partner using only haptic cues for intent inference. At the upper tier, a lightweight behavior-cloning network consumes six-axis force/torque streams from dual wrist-mounted sensors and outputs whole-body planar velocity commands that capture the leader's applied forces. At the lower tier, a deep-reinforcement-learning policy, trained under randomized payloads (0-3 kg) and friction conditions in Isaac Gym and validated in MuJoCo and on a real Unitree G1, maps these high-level twists to stable, under-load joint trajectories. By decoupling intent interpretation (force -> velocity) from legged locomotion (velocity -> joints), our method combines intuitive responsiveness to human inputs with robust, load-adaptive walking. We collect training data without motion-capture or markers, only synchronized RGB video and F/T readings, employing SAM2 and WHAM to extract 3D human pose and velocity. In real-world trials, our humanoid achieves cooperative carry-and-move performance (completion time, trajectory deviation, velocity synchrony, and follower-force) on par with a blindfolded human-follower baseline. This work is the first to demonstrate learned haptic guidance fused with full-body legged control for fluid human-humanoid co-manipulation. Code and videos are available on the H2-COMPACT website.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。