用粗到精框架提升人体骨骼与表面的生物力学还原精度
SKEL-CF: Coarse-to-Fine Biomechanical Skeleton and Surface Mesh Recovery
- 分阶段优化:先粗略估计再逐步细化骨骼参数和相机位姿
- 在MOYO数据集上达到85.0 MPJPE/51.4 PA-MPJPE,优于前代方法
- 适用于需要高生物力学真实性的运动分析场景
参数化3D人体模型如SMPL推动了姿态与体型估计的发展,但其简化的运动学限制了生物力学真实性。近期提出的SKEL模型通过解构SMPL并替换为解剖学准确的骨架,解决了该问题。然而,直接估计SKEL参数仍面临训练数据有限、视角模糊及人体关节复杂性等挑战。本文提出SKEL-CF,一种从粗到精的框架用于SKEL参数估计。该框架采用基于Transformer的编码器-解码器结构,编码器预测粗粒度的相机与SKEL参数,解码器在多层中逐步细化。为确保解剖一致性监督,我们将现有SMPL数据集4DHuman转换为对齐的4DHuman-SKEL版本,提供高质量训练数据。此外,为缓解深度与尺度模糊,显式引入相机建模,并验证其在多视角下的重要性。大量实验验证了设计有效性。在挑战性数据集MOYO上,SKEL-CF取得85.0 MPJPE / 51.4 PA-MPJPE,显著优于此前基于SKEL的最优方法HSMR(104.5 / 79.6)。结果表明SKEL-CF是可扩展且解剖学忠实的人体运动分析框架,助力计算机视觉技术在生物力学分析中的应用。代码已开源。
原文摘要 · Abstract (English)
Parametric 3D human models such as SMPL have driven significant advances in human pose and shape estimation, yet their simplified kinematics limit biomechanical realism. The recently proposed SKEL model addresses this limitation by re-rigging SMPL with an anatomically accurate skeleton. However, estimating SKEL parameters directly remains challenging due to limited training data, perspective ambiguities, and the inherent complexity of human articulation. We introduce SKEL-CF, a coarse-to-fine framework for SKEL parameter estimation. SKEL-CF employs a transformer-based encoder-decoder architecture, where the encoder predicts coarse camera and SKEL parameters, and the decoder progressively refines them in successive layers. To ensure anatomically consistent supervision, we convert the existing SMPL-based dataset 4DHuman into a SKEL-aligned version, 4DHuman-SKEL, providing high-quality training data for SKEL estimation. In addition, to mitigate depth and scale ambiguities, we explicitly incorporate camera modeling into the SKEL-CF pipeline and demonstrate its importance across diverse viewpoints. Extensive experiments validate the effectiveness of the proposed design. On the challenging MOYO dataset, SKEL-CF achieves 85.0 MPJPE / 51.4 PA-MPJPE, significantly outperforming the previous SKEL-based state-of-the-art HSMR (104.5 / 79.6). These results establish SKEL-CF as a scalable and anatomically faithful framework for human motion analysis, facilitating the use of computer vision techniques in biomechanics-related analysis. Our implementation is available on the project page: https://pokerman8.github.io/SKEL-CF/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。