用机器人模拟婴儿动作与感官体验,助力发育研究与早筛。
Simulating Infant First-Person Sensorimotor Experience via Motion Retargeting from Babies to Humanoids

- 从视频重建婴儿姿态,映射到物理/虚拟机器人
- 实现亚厘米级精度,生成触觉、本体感觉和视觉流
- 适合发展科学、机器人学及神经发育障碍研究
随着类人机器人能力提升,将人类运动迁移到类人代理越来越重要。然而现有方法多关注运动学再现,忽略伴随的丰富感知运动体验。本文提出一种框架,利用物理与虚拟类人机器人模拟婴儿多模态感知运动体验。仅需单段视频,方法通过提取骨骼结构并估计每帧完整3D姿态来重建婴儿身体构型,再将运动映射至多个发育平台:物理iCub机器人与虚拟仿真器pyCub、EMFANT和MIMo。在这些实体上重放迁移后的动作,生成包含本体感觉(关节与肌肉)、触觉和视觉的模拟多感官流。对于最佳匹配实体,运动迁移达到亚厘米级精度,支持婴儿发育的丰富多模态分析,并提升行为自动标注效果。该框架为理解婴儿感知运动体验提供了新视角,为机器人学、发育科学及神经发育障碍早期检测提供新工具。代码已开源:https://github.com/ctu-vras/motion-retargeting/
原文摘要 · Abstract (English)
Motion retargeting from humans to human-like artificial agents is becoming increasingly important as humanoid robots grow more capable. However, most existing approaches focus only on reproducing kinematics and ignore the rich sensorimotor experience associated with human movement. In this work, we present a framework for simulating the multimodal sensorimotor experiences of infants using physical and virtual humanoids. From a single video, our method reconstructs the infant's body configuration by extracting its skeletal structure and estimating the full 3D pose from each frame. Then we map the reconstructed motion onto several developmental platforms: the physical iCub robot and the virtual simulators pyCub, EMFANT and MIMo. Replaying the retargeted motions on these embodiments produces simulated multisensory streams including proprioception (joints and muscles), touch, and vision. For the best-matching embodiment, the retargeting achieves sub-centimeter accuracy and enables a rich multimodal analysis of infant development as well as enhanced automated annotation of behaviors. This framework provides a unique window into the infant's sensorimotor experience, offering new tools for robotics, developmental science, and early detection of neurodevelopmental disorders. The code is available at https://github.com/ctu-vras/motion-retargeting/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。