用眼周关键点引导注意力，动态调整匹配阈值，提升口罩人脸识别准确率。
PLGSA-Transformer: Periocular Landmark-Guided Attention with Occlusion-Adaptive Cosine Thresholding for Cross-Modal Masked and Unmasked Face Recognition
- 基于眼周关键点生成热图，引导模型关注可见面部区域。
- 引入自适应阈值机制，遮挡越严重匹配门槛越高，准确率达97.22%。
- 适合安防、身份验证等真实场景中带口罩的人脸识别应用。
新冠疫情推动口罩普及，传统人脸识别系统在面部被遮挡时性能大幅下降。现有方法依赖固定余弦阈值、非自适应卷积网络和纯数据驱动特征，难以泛化。本文提出PLGSA-Transformer跨模态人脸识别框架：第一，通过MediaPipe关键点生成眼区、眉区和额区高斯热图，结合EfficientNetB3特征，经可学习残差门融合，实现空间注意力引导；第二，采用混合CNN-Transformer分支，将特征图转为令牌，由双层多头自注意编码器建模跨区域依赖关系；第三，提出联合训练的遮挡自适应余弦阈值（OACT）模块，根据预测遮挡程度动态提高匹配阈值。模型在Zenodo MDMFR（60%）、Kaggle CelebA-HQ掩码数据集（25%）及作者收集数据（15%）上评估，涵盖21-75岁不同性别与口罩类型，使用统一损失函数（对比验证、身份分类、遮挡交叉熵）。最终达到97.22%配对验证准确率，ROC AUC为1.0000，显著优于VGG-16基线MUFM（95.0%）、HOG分类器（85.0%）和基于结构度量的方法（86.61%）。结果表明，将眼周几何信息融入注意力机制，结合Transformer建模与遮挡自适应阈值，可实现鲁棒、可扩展的跨模态口罩人脸识别。
原文摘要 · Abstract (English)
The widespread adoption of facial masks, accelerated by COVID-19 and mandated in security-sensitive settings, has exposed limitations of conventional face recognition systems. Existing approaches relying on fixed cosine thresholds, non-adaptive CNNs, and purely data-driven features fail to generalize when facial regions are occluded, creating a gap between lab performance and real-world deployability. This paper proposes PLGSA-Transformer, a cross-modal face matching framework with three contributions. First, Periocular Landmark-Guided Spatial Attention (PLGSA) uses MediaPipe landmarks to compute Gaussian heatmaps over the eye, brow, and forehead regions, fusing them with EfficientNetB3 features via a learnable residual gate to direct attention toward discriminative visible regions. Second, a Hybrid CNN-Transformer Branch reshapes feature maps into tokens processed by a two-layer Multi-Head Self-Attention encoder, enabling cross-regional dependency modelling. Third, the Occlusion-Adaptive Cosine Threshold (OACT) is a jointly trained head that raises the matching threshold in proportion to predicted occlusion severity. The model is evaluated on 858 images from Zenodo MDMFR (60%), Kaggle CelebA-HQ masked collection (25%), and author-collected images (15%), spanning both genders, ages 21-75, with varied mask types, trained via a unified loss combining contrastive verification, identity classification, and occlusion cross-entropy. PLGSA-Transformer achieves 97.22% pair verification accuracy with ROC AUC 1.0000, surpassing VGG-16-based MUFM (Abdullah et al., 2025; 95.0%), HOG classifiers (Adnan et al., 2020; 85.0%), and Feature-based Structural Measure (Shnain et al., 2017; 86.61%). These results confirm that encoding periocular geometry into attention, with Transformer modelling and occlusion-adaptive thresholds, yields a robust, scalable solution for cross-modal masked face recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。