首次系统分析3D手部姿态估计中合成数据与真实数据的差距，找出关键影响因素。
Analyzing the Synthetic-to-Real Domain Gap in 3D Hand Pose Estimation
- 通过构建高质量合成数据流水线，系统分析合成到真实的数据差距
- 发现手臂、图像频谱统计、手部姿势和物体遮挡是主要差异来源
- 整合关键因素后，合成数据可达到真实数据的准确率，实现纯合成训练
近年来，用于面部、身体和手部的合成3D人体数据集在逼真度上取得显著进展。面部识别和身体姿态估计已仅用合成数据达到顶尖性能，但手部仍存在显著的合成到真实差距。本文首次对3D手部姿态估计中的合成-真实域差距进行系统研究，分析并识别出关键影响因素，包括前臂、图像频率统计、手部姿态和物体遮挡。为支持分析，我们提出一种数据合成流水线以生成高质量数据。结果表明，当整合这些关键成分后，合成手部数据可达到与真实数据相当的精度，为纯合成数据训练手部姿态估计铺平道路。代码与数据见：https://github.com/delaprada/HandSynthesis.git。
原文摘要 · Abstract (English)
Recent synthetic 3D human datasets for the face, body, and hands have pushed the limits on photorealism. Face recognition and body pose estimation have achieved state-of-the-art performance using synthetic training data alone, but for the hand, there is still a large synthetic-to-real gap. This paper presents the first systematic study of the synthetic-to-real gap of 3D hand pose estimation. We analyze the gap and identify key components such as the forearm, image frequency statistics, hand pose, and object occlusions. To facilitate our analysis, we propose a data synthesis pipeline to synthesize high-quality data. We demonstrate that synthetic hand data can achieve the same level of accuracy as real data when integrating our identified components, paving the path to use synthetic data alone for hand pose estimation. Code and data are available at: https://github.com/delaprada/HandSynthesis.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。