通过统一域提升3D人体姿态估计泛化能力,免去目标域微调。
Toward Efficient Generalization in 3D Human Pose Estimation via a Canonical Domain Approach
- 将源域与目标域映射到统一标准域,实现跨域一致的2D-3D姿态映射。
- 在Human3.6M、Fit3D等数据集上,相同数据量下显著提升跨域泛化性能。
- 适合需要快速部署到新场景且无标注数据的3D姿态估计应用。
深度学习虽显著提升了3D人体姿态估计(HPE)性能,但源域与目标域间的域差异仍导致性能下降,通常需大量数据增强或针对每个目标域微调。为此,本文提出一种新的标准域方法,将源域与目标域统一映射至一个标准域,避免在目标域进行额外微调。通过引入标准化过程,构建新型2D-3D姿态映射,确保2D-3D姿态一致性并简化模式,从而提升提升网络训练效率。标准域的构建包括:(1) 在源域中,提升网络在标准域内训练;(2) 在目标域中,利用透视投影性质与已知相机内参,对输入2D姿态进行标准化后再推理。实验在多种提升网络及公开数据集(如Human3.6M、Fit3D、MPI-INF-3DHP)上验证,该方法在使用相同数据量条件下,显著提升跨数据集泛化能力。
原文摘要 · Abstract (English)
Recent advancements in deep learning methods have significantly improved the performance of 3D Human Pose Estimation (HPE). However, performance degradation caused by domain gaps between source and target domains remains a major challenge to generalization, necessitating extensive data augmentation and/or fine-tuning for each specific target domain. To address this issue more efficiently, we propose a novel canonical domain approach that maps both the source and target domains into a unified canonical domain, alleviating the need for additional fine-tuning in the target domain. To construct the canonical domain, we introduce a canonicalization process to generate a novel canonical 2D-3D pose mapping that ensures 2D-3D pose consistency and simplifies 2D-3D pose patterns, enabling more efficient training of lifting networks. The canonicalization of both domains is achieved through the following steps: (1) in the source domain, the lifting network is trained within the canonical domain; (2) in the target domain, input 2D poses are canonicalized prior to inference by leveraging the properties of perspective projection and known camera intrinsics. Consequently, the trained network can be directly applied to the target domain without requiring additional fine-tuning. Experiments conducted with various lifting networks and publicly available datasets (e.g., Human3.6M, Fit3D, MPI-INF-3DHP) demonstrate that the proposed method substantially improves generalization capability across datasets while using the same data volume.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。