用2D数据预训练，提升3D人体姿态估计的泛化能力与效率
2D Pre-Training for 3D Pose Estimation
- 在2D人体姿态数据上预训练模型，再迁移到3D任务
- 结合MPII和Human3.6M数据，实现64.5mm以下的MPJPE指标
- 小规模模型也能通过2D预训练获得良好泛化性能
预训练是深度学习中广泛应用的方法，先在某一任务上训练模型，再在下游任务上微调，有助于模型学习更通用的输入表征。尽管已有研究将预训练用于3D人体姿态估计（3D HPE），但通常仅限于少数强基准数据集（如Human3.6M）。本文扩展了现有3D HPE方案，使其兼容更多2D和3D HPE数据集，包括Occlusion Person。我们系统研究了2D预训练中模型规模等因素对下游性能的影响，并评估其在不同数据集间的泛化能力。实验表明，2D预训练显著优于仅在3D数据上训练，尤其在计算效率方面表现更优。最终，在MPII和Human3.6M数据集上，实现了低于64.5mm的平均关节位置误差（MPJPE）。
原文摘要 · Abstract (English)
Pre-training is a general method that is used in a range of deep learning tasks. By first training a model on one task, and then further training on the downstream task used for final evaluation, the model is forced to learn a more general understanding of the input data. While pre-training has been applied to 3D Human Pose Estimation (HPE) previously, the scope of datasets used is typically very limited to some strong benchmarks, like Human3.6M. Therefore, in this project, we expand the scope of an existing 3D HPE scheme to be compatible with additional 2D and 3D HPE datasets, like Occlusion Person. We perform an extensive study on how aspects of 2D pre-training, such as model size, affect downstream performance, and to what extent pre-training can help the model generalize to different datasets. Experimental results show that 2D pre-training consistently outperforms training on 3D data alone, particularly in terms of computational efficiency. Finally, using MPII and Human3.6M, we are able to obtain an MPJPE score of under 64.5mm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。