构建首个自拍视角4D人地形行走数据集，助力机器人学会在复杂地形中自然移动。
EgoHTR: Egocentric 4D Demonstrations of Human Terrain Traversal
- 用可穿戴设备与便携3D扫描仪采集55段自拍视角4D人体运动序列
- 数据含超15万帧，对比动捕真值达到顶尖精度，建立新基准
- 已用于训练感知式行走策略并实现在Unitree G1上的部署
将人形机器人部署于非结构化地形仍是未解难题。传统强化学习难以应对真实世界交互的复杂性，而依赖人类先验的更优方法又受限于缺乏情境感知的模型。现有数据集管道无法捕捉挑战环境中的人-场景动态序列，导致运动生成受限。为此，我们提出自拍视角人地形重建（EgoHTR）数据集，开发并开源一套重建流程，使用多传感器组合（自拍可穿戴设备+便携3D扫描仪），在多样复杂环境中共采集55段场景对齐的4D人体运动序列。数据集共包含超过15万帧，经与动作捕捉真值对比，展现出顶尖精度，确立了人类运动分析与合成的新基准。进一步利用该数据训练感知式行走策略，并成功在Unitree G1上实现参考运动的硬件部署。该流程支持社区扩展，有助于研究人员构建具备情境感知能力、可靠穿越不平地形的机器人基础系统。
原文摘要 · Abstract (English)
Deploying humanoid robots in unstructured terrain remains an open problem. While classic reinforcement learning struggles with the sheer complexity of real-world interactions, more promising methods leveraging human priors remain limited to models lacking contextual awareness. The restricted motion synthesis is a direct consequence of existing dataset pipelines failing to capture human-scene sequences in challenging environments. To bridge this gap between humanoid learning and scene reconstruction, we introduce the Egocentric Human-Terrain Reconstruction (EgoHTR) dataset. We develop and open-source a reconstruction pipeline capturing 55 scene-aligned 4D human motion sequences in diverse, complex environments using a multi-sensor setup of egocentric wearables and a portable 3D scanner. The resulting dataset comprises over 150k frames, which we evaluate against motion-capture ground truth, demonstrating state-of-the-art accuracy and establishing a rigorous benchmark for human motion analysis and synthesis. Further, we leverage this data to train perceptive locomotion policies, demonstrating hardware deployment on a Unitree G1 for reconstructed reference motions. Our pipeline enables community-driven dataset extensions and factors the problem to help researchers build foundational, context-aware robots that reliably traverse uneven terrain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。