用大模型提升历史画像人脸识别准确率
ArtFace: Towards Historical Portrait Face Identification via Model Adaptation
- 融合大模型与传统识别网络的嵌入特征
- 在多个艺术画像数据集上超越现有最佳方法
- 适合艺术史研究与跨域人脸识别场景
识别历史绘画中的人物是艺术史研究的关键，有助于理解人物生平及其自我呈现方式。然而，该过程常具主观性，受限于数据匮乏和风格差异。自动人脸识別虽能应对复杂条件，但传统模型在照片上表现良好，在绘画上因领域差异和类内变化大而效果不佳。艺术风格、技巧、意图及对其他作品的影响进一步增加难度。本文研究基础模型在艺术人脸识别中的潜力，通过微调基础模型并融合其嵌入特征与传统人脸识别网络，显著优于当前最先进方法。结果表明，基础模型可有效弥合传统方法在该任务中的失效空白。
原文摘要 · Abstract (English)
Identifying sitters in historical paintings is a key task for art historians, offering insight into their lives and how they chose to be seen. However, the process is often subjective and limited by the lack of data and stylistic variations. Automated facial recognition is capable of handling challenging conditions and can assist, but while traditional facial recognition models perform well on photographs, they struggle with paintings due to domain shift and high intra-class variation. Artistic factors such as style, skill, intent, and influence from other works further complicate recognition. In this work, we investigate the potential of foundation models to improve facial recognition in artworks. By fine-tuning foundation models and integrating their embeddings with those from conventional facial recognition networks, we demonstrate notable improvements over current state-of-the-art methods. Our results show that foundation models can bridge the gap where traditional methods are ineffective. Paper page at https://www.idiap.ch/paper/artface/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。