构建毛孔级图像数据集，评估深度学习描述子在人脸弱纹理区域的表现
Pore-scale Image Patch Dataset and A Comparative Evaluation of Pore-scale Facial Features
- 提出数据-模型协同进化框架，生成高质量毛孔级图像数据集
- SOTA模型在匹配任务中FPR95达1.91%，优于PSIFT的22.41%
- 深度描述子在3D重建中优势不明显，仍难应对人脸弱纹理挑战
人脸皮肤区域纹理较弱，给面部运动分析和三维重建中的局部描述子匹配带来挑战。尽管基于深度学习的描述子在诸多应用中表现优于传统手工设计描述子，但毛孔级图像块数据集的缺乏制约了其在人脸领域的进一步发展。本文提出PorePatch数据集，一个高质量的毛孔级图像块数据集，并建立合理评估基准。我们引入数据-模型协同进化（DMCE）框架，从高分辨率人脸图像中逐步生成高质量数据集。随后在该数据集上训练现有最先进模型并开展广泛实验。结果表明，最先进模型在匹配任务中达到1.91%的FPR95，显著优于PSIFT的22.41%。然而，在3D重建任务中，其整体性能并未显著优于传统描述子，表明深度学习描述子在处理人脸弱纹理区域方面仍存在局限，该领域仍有大量工作待完成。
原文摘要 · Abstract (English)
The weak-texture nature of facial skin regions presents significant challenges for local descriptor matching in applications such as facial motion analysis and 3D face reconstruction. Although deep learning-based descriptors have demonstrated superior performance to traditional hand-crafted descriptors in many applications, the scarcity of pore-scale image patch datasets has hindered their further development in the facial domain. In this paper, we propose the PorePatch dataset, a high-quality pore-scale image patch dataset, and establish a rational evaluation benchmark. We introduce a Data-Model Co-Evolution (DMCE) framework to generate a progressively refined, high-quality dataset from high-resolution facial images. We then train existing SOTA models on our dataset and conduct extensive experiments. Our results show that the SOTA model achieves a FPR95 value of 1.91% on the matching task, outperforming PSIFT (22.41%) by a margin of 20.5%. However, its advantage is diminished in the 3D reconstruction task, where its overall performance is not significantly better than that of traditional descriptors. This indicates that deep learning descriptors still have limitations in addressing the challenges of facial weak-texture regions, and much work remains to be done in this field.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。