用专家混合模型提升低分辨率人脸识别准确率。
FaceMoE: Mixture of Experts for Low-Resolution Face Recognition

- 引入动态路由的专家网络,实现人脸不同区域的针对性特征提取。
- 在11个数据集上超越现有方法,尤其在极端模糊和低对比度下表现突出。
- 适合需要高精度低分辨率人脸识别的应用场景,如安防监控。
低分辨率人脸识别(LR-FR)因特征提取与聚合困难而具有挑战性,探针图像常因模糊、遮挡、低对比度等极端退化导致身份信息匮乏。此外,高分辨率(HR)图库与低分辨率(LR)探针间存在显著域差距。单一特征编码器在微调低分辨率数据集时难以跨域泛化,且易产生灾难性遗忘。为此,我们提出FaceMoE,一种面向低分辨率人脸识别的专家混合(MoE)Transformer架构。通过引入多个专用前馈网络(FFN)专家与top-k路由器,动态分配令牌至合适专家,促进各专家对人脸不同语义区域的专精化,实现分辨率达标的特征提取。同时,稀疏激活机制在不增加计算开销的前提下提升模型容量,保留预训练知识。采用联合人脸识别损失、路由器z-loss与负载均衡损失训练,确保专家专精与训练稳定。据我们所知,这是首个将MoE应用于LR-FR的工作。在涵盖高分辨率、混合质量与低分辨率基准的11个数据集上进行的大量实验表明,FaceMoE显著优于现有最先进方法。
原文摘要 · Abstract (English)
Low-resolution face recognition (LR-FR) remains a challenging task due to poor feature extraction and aggregation, as probe images often contain limited identity information resulting from extreme degradations such as blur, occlusion, and low contrast. Additionally, the domain gap between high-resolution (HR) gallery images and low-resolution (LR) probe images poses a significant challenge. A single feature encoder struggles to generalize effectively across both domains when fine-tuned on an LR dataset, and this issue is further magnified by catastrophic forgetting. To address these challenges, we propose FaceMoE, an effective adaptation of Mixture of Experts (MoE) transfomer architecture for low-resolution face-recognition . Specifically, we introduce multiple specialized feed-forward network (FFN) experts and incorporate a top-k router, which dynamically assigns tokens to appropriate experts. This design emergently promotes specialization across experts for different semantic regions of the face, which enables FaceMoE to perform resolution-aware feature extraction. Moreover, the top-k router facilitates sparse expert activation, enabling the model to preserve pretrained knowledge when finetuned on a LR dataset, while increasing model capacity without proportional computational overhead. FaceMoE is trained with a combined face recognition loss, router z-loss, and load balancing loss to ensure expert specialization and stable training. To the best of our knowledge, this is the first work leveraging MoE for LR-FR. Extensive experiments across eleven datasets, spanning HR, mixed-quality, and LR benchmarks, demonstrate that FaceMoE significantly outperforms state-of-the-art methods. Code: https://github.com/Kartik-3004/FaceMoE
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。