用真实+合成动作数据微调模型，提升跳马运动极端姿势的估计精度
Human Pose Estimation in Trampoline Gymnastics: How to Improve Performance on Extreme Poses
- 结合真实与合成极端动作数据微调ViTPose模型
- 3D关键点误差降低46.1毫米，提升42.7%
- 适合需要高精度姿态估计的极限运动分析场景
跳马运动包含极端人体姿态和非常见视角，现有先进姿态估计算法在此类数据上表现不佳。本文通过在真实极端姿态与领域特定合成姿态（STP）组合数据上微调姿态估计算法，有效改善性能。我们利用跳马动作的动作捕捉数据生成多视角高保真图像及精确关键点标签，构建合成数据管道。在真实多视角图像上测试，微调后的ViTPose模型在2D估计上达到与MS COCO验证集上最先进模型相当的水平，且适用于更具挑战性的数据。在3D重建中，最优模型将MPJPE降低46.1毫米，相比预训练的ViTPose提升42.7%。代码与数据已开源。
原文摘要 · Abstract (English)
Trampoline gymnastics involves extreme human poses and uncommon viewpoints, on which state-of-the art pose estimation models tend to under-perform. We demonstrate that this problem can be addressed by fine-tuning a pose estimation model on a combination of real extreme poses and domain-specific synthetic poses (STP). We generate STP from motion capture recordings of trampoline routines. We propose a pipeline to fit noisy motion capture data to a parametric human model, then generate multi-view realistic images with high-fidelity keypoint labels. The fine-tuned ViTPose model tested on real multi-view images exhibits accuracy improvements in 2D which translate to improved 3D triangulation. In 2D, we obtain a performance similar to state-of-the-art models on the MS COCO validation set while evaluating on significantly more challenging data, bridging the performance gap between common and extreme poses. In 3D, we reduce the MPJPE by 46.1 mm with our best model, which represents an improvement of 42.7% compared to the pretrained ViTPose model. Our code and data are available at https://github.com/VisionICLab/trampoline_syn_data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。