仅用三个传感器实现全身姿态实时估计,适合虚拟现实应用。
Progressive Inertial Poser: Progressive Real-Time Kinematic Chain Estimation for 3D Full-Body Pose from Three IMU Sensors
- 结合神经网络与人体动力学模型,分阶段逐步重建全身动作。
- 在多个公开数据集上表现优于同配置方法,接近六传感器水平。
- 仅需头和双手佩戴传感器,硬件部署更简单实用。
支持全身虚拟表征的动作捕捉系统对虚拟现实至关重要。相较于基于视觉的系统,仅依赖稀疏信号的全身姿态估计不受环境或拍摄范围限制。然而,以往方法要么需要在骨盆和下肢额外佩戴传感器,要么依赖外部视觉传感器获取关键关节全局位置。为提升技术在虚拟现实中的实用性,本文仅使用佩戴于头部和双腕的三个惯性测量单元(IMU)的惯性数据,实现全身姿态估计,降低系统复杂度。提出一种名为渐进式惯性姿态解算器(ProgIP)的方法,融合神经网络与人体动力学模型,考虑运动链层级结构,并采用多阶段渐进式网络加深特征提取,实现实时全身体态重建。编码器结合Transformer Encoder与双向LSTM(TE-biLSTM),灵活捕捉惯性序列的时间依赖性;解码器基于多层感知机(MLPs),将高维特征准确映射到皮肤化多人线性(SMPL)模型参数。在多个公开数据集上的定量与定性实验表明,本方法在相同输入条件下优于当前最优方法,且性能接近使用六个IMU传感器的最新工作。
原文摘要 · Abstract (English)
The motion capture system that supports full-body virtual representation is of key significance for virtual reality. Compared to vision-based systems, full-body pose estimation from sparse tracking signals is not limited by environmental conditions or recording range. However, previous works either face the challenge of wearing additional sensors on the pelvis and lower-body or rely on external visual sensors to obtain global positions of key joints. To improve the practicality of the technology for virtual reality applications, we estimate full-body poses using only inertial data obtained from three Inertial Measurement Unit (IMU) sensors worn on the head and wrists, thereby reducing the complexity of the hardware system. In this work, we propose a method called Progressive Inertial Poser (ProgIP) for human pose estimation, which combines neural network estimation with a human dynamics model, considers the hierarchical structure of the kinematic chain, and employs a multi-stage progressive network estimation with increased depth to reconstruct full-body motion in real time. The encoder combines Transformer Encoder and bidirectional LSTM (TE-biLSTM) to flexibly capture the temporal dependencies of the inertial sequence, while the decoder based on multi-layer perceptrons (MLPs) transforms high-dimensional features and accurately projects them onto Skinned Multi-Person Linear (SMPL) model parameters. Quantitative and qualitative experimental results on multiple public datasets show that our method outperforms state-of-the-art methods with the same inputs, and is comparable to recent works using six IMU sensors.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。