提升无标记单目视频中足部运动重建精度
Improving 3D Foot Motion Reconstruction in Markerless Monocular Human Motion Capture
- 通过2D足关节点序列升维重构3D足部动作
- 在MOYO数据集上将踝关节角度误差降低30%
- 适合需要精细足部动作的动画与步态分析
现有先进方法可从野外视频中恢复准确的整体3D人体动作，但在足部等细微关节运动上表现不佳，影响步态分析和动画应用。原因在于训练数据存在足部标注不准确且运动多样性不足。为此提出FootMR，一种足部运动精修方法，通过将已有模型估计的2D足部关键点序列提升至3D来优化足部动作。该方法避免直接使用图像输入，从而避开图像-3D标注对不一致的问题，转而利用大规模动捕数据。为解决2D到3D映射中的歧义性，FootMR引入膝部和足部运动作为上下文信息，仅预测残差足部运动。通过采用全局而非父相对旋转表示关节，并结合大量数据增强，显著提升对极端足部姿态的泛化能力。为评估足部运动重建效果，引入MOOF——一个包含复杂足部动作的2D数据集。在MOOF、MOYO和RICH上的实验表明，FootMR优于现有方法，在MOYO上将踝关节角度误差相比最佳视频基线降低30%。
原文摘要 · Abstract (English)
State-of-the-art methods can recover accurate overall 3D human body motion from in-the-wild videos. However, they often fail to capture fine-grained articulations, especially in the feet, which are critical for applications such as gait analysis and animation. This limitation results from training datasets with inaccurate foot annotations and limited foot motion diversity. We address this gap with FootMR, a Foot Motion Refinement method that refines foot motion estimated by an existing human recovery model through lifting 2D foot keypoint sequences to 3D. By avoiding direct image input, FootMR circumvents inaccurate image-3D annotation pairs and can instead leverage large-scale motion capture data. To resolve ambiguities of 2D-to-3D lifting, FootMR incorporates knee and foot motion as context and predicts only residual foot motion. Generalization to extreme foot poses is further improved by representing joints in global rather than parent-relative rotations and applying extensive data augmentation. To support evaluation of foot motion reconstruction, we introduce MOOF, a 2D dataset of complex foot movements. Experiments on MOOF, MOYO, and RICH show that FootMR outperforms state-of-the-art methods, reducing ankle joint angle error on MOYO by up to 30% over the best video-based approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。