用单目摄像头让半人形机器人模仿人类手臂动作
Imitation of Arm Gestures by the Semi-Humanoid Robot NICO
- 基于几何关系与MediaPipe模型提取人体关节3D坐标
- 仅凭单张图像实现六名参与者不同身高的动作模仿
- 适合研究人机交互与机器人动作生成的开发者
无缝人机交互需机器人具备感知与运动能力，其中模仿人类手势尤为重要。半人形机器人NICO因具类人特征，在人机交互中具有优势。本文提出一种基于解析几何与预训练MediaPipe姿态估计模型的臂部动作模仿系统。对每帧输入的RGB图像，利用MediaPipe框架获取人体相关关键点（包括手臂关节与手部关键点）的3D坐标，再通过推导的几何关系计算关节角度，最终将角度映射至NICO的电机配置，并按预设序列执行动作。六名不同身高的参与者在多个典型手臂动作上的初步实验表明，该方法仅需单目RGB输入即可生成有意义的模仿动作，但对复杂姿态和手腕动作仍存在局限。
原文摘要 · Abstract (English)
Seamless human-robot interaction (HRI) requires a number of perceptual and motor abilities from the robot, one of them being the imitation of human gestures. Humanoid robots have an advantage in HRI thanks to their anthropomorphic features. In this work, we develop a system for imitation of human arm gestures by the semi-humanoid robot NICO based on analytical geometry and a pretrained MediaPipe pose-estimation model. For each input RGB frame, 3D coordinates of relevant human body landmarks, including arm joints and hand keypoints, are obtained using the MediaPipe framework. Joint angles are then computed from these coordinates using derived geometric relations. Finally, the computed angles are properly mapped to NICO's motor configuration and executed in a predefined motion sequence. Preliminary experiments on several representative arm gestures with six participants of different height indicate that the proposed method can produce meaningful imitative motions from monocular RGB input only, while also highlighting limitations in more complex poses and wrist-related movements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。