用可微分PnP实现端到端人脸重建，精度领先
End to End Face Reconstruction via Differentiable PnP
- 双分支网络分别负责3D人脸重建与2D关键点检测
- 在MVP-Human数据集上取得优异定量结果
- 适合需要高精度3D人脸建模的研究者
本文是ECCV 2022 WCPA挑战赛中人脸重建赛道的挑战报告。我们设计了一个双分支网络，分别完成3D人脸重建和2D关键点检测任务。前者输出规范化的3D人脸坐标，后者输出像素坐标，即3D坐标在头部姿态和透视投影下的映射。此外，我们引入可微分的PnP（Perspective-n-Points）层对两分支输出进行联合优化。该方法在MVP-Human数据集上表现优异，获得挑战赛第三名。
原文摘要 · Abstract (English)
This is a challenge report of the ECCV 2022 WCPA Challenge, Face Reconstruction Track. Inside this report is a brief explanation of how we accomplish this challenge. We design a two-branch network to accomplish this task, whose roles are Face Reconstruction and Face Landmark Detection. The former outputs canonical 3D face coordinates. The latter outputs pixel coordinates, i.e. 2D mapping of 3D coordinates with head pose and perspective projection. In addition, we utilize a differentiable PnP (Perspective-n-Points) layer to finetune the outputs of the two branch. Our method achieves very competitive quantitative results on the MVP-Human dataset and wins a $3^{rd}$ prize in the challenge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。