用手机手表等设备的传感器实时估算全身动作和空间位置。
MobilePoser: Real-Time Full-Body Pose Estimation and 3D Human Translation from IMUs in Mobile Consumer Devices
- 分阶段深度网络+物理优化，实现低延迟姿态估计。
- 无需额外设备，可在消费级硬件上实时运行。
- 适合健康监测、游戏交互、室内导航等场景。
全身体感捕捉正从专用设备向集成在手机、手表、耳机等消费级设备中的低精度惯性传感器演进。然而，随着传感器精度下降，存在在线性能差、时间不连续及全局位移丢失等问题。为此，我们提出 MobilePoser，一种基于任意可用组合惯性测量单元（IMUs）的实时全身姿态与全局位移估计算法。系统采用多阶段深度神经网络进行运动学姿态估计，再通过基于物理的运动优化器提升精度，实现了当前最优性能且模型轻量。我们展示了多个应用实例，涵盖健康与健身、游戏交互、室内导航等领域，凸显其广泛应用潜力。
原文摘要 · Abstract (English)
There has been a continued trend towards minimizing instrumentation for full-body motion capture, going from specialized rooms and equipment, to arrays of worn sensors and recently sparse inertial pose capture methods. However, as these techniques migrate towards lower-fidelity IMUs on ubiquitous commodity devices, like phones, watches, and earbuds, challenges arise including compromised online performance, temporal consistency, and loss of global translation due to sensor noise and drift. Addressing these challenges, we introduce MobilePoser, a real-time system for full-body pose and global translation estimation using any available subset of IMUs already present in these consumer devices. MobilePoser employs a multi-stage deep neural network for kinematic pose estimation followed by a physics-based motion optimizer, achieving state-of-the-art accuracy while remaining lightweight. We conclude with a series of demonstrative applications to illustrate the unique potential of MobilePoser across a variety of fields, such as health and wellness, gaming, and indoor navigation to name a few.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。