通过动态调整面部关键点与视觉语言信息，提升复杂环境下表情识别的准确性。
LaCoVL-FER: Landmark-Guided Contrastive Learning Network with Vision-Language Enhancement for Facial Expression Recognition
- 基于关键点引导的自适应编码器，动态校准局部特征。
- 在RAF-DB等数据集上达到新最优，准确率提升显著。
- 适合需要高鲁棒性表情识别的应用场景。
野外环境下的面部表情识别（FER）需在姿态、遮挡、光照和身份差异下识别细微表情线索。现有方法通过引入视觉注意力、面部关键点或视觉语言模型作为辅助先验来提升鲁棒性，但这些先验通常静态融合，无法捕捉个体差异，导致注意力冗余和表征不稳定。为此，我们提出LaCoVL-FER：一种基于关键点引导的对比学习网络，并结合视觉语言增强，将FER从静态先验注入转变为样本自适应先验优化。具体地，一个关键点引导的自适应编码器（LGAE）通过双分支门控交叉注意力（BGCA）利用关键点几何结构校准区域外观特征，抑制噪声响应，生成表达相关表征。同时，视觉语言增强策略（VLES）将冻结的CLIP图像编码器的通用视觉特征，转化为表达特异性视觉表示。基于此，表达条件提示机制（ECP）将冻结的CLIP文本编码器中的固定类别文本提示，动态调整为实例感知的文本表示。最终的视觉-文本表征被对齐为自适应语义先验，增强模型鲁棒性与泛化能力。定量与定性实验表明，LaCoVL-FER在RAF-DB、FERPlus和AffectNet上均优于现有最先进方法。代码已开源：https://github.com/ylin06804/LaCoVL-FER。
原文摘要 · Abstract (English)
Facial Expression Recognition (FER) in the wild requires models to identify subtle expression cues under large variations in pose, occlusion, illumination, and identity. Recent FER methods improve robustness by introducing visual attention, facial landmarks, or vision-language models as auxiliary priors. However, these priors are typically integrated in a static manner, failing to capture instance-specific facial variations, thereby resulting in severe attention redundancy and representation instability. To address this issue, we propose LaCoVL-FER, a landmark-guided contrastive learning network with vision-language enhancement for FER, which shifts FER from static prior injection to sample-adaptive prior refinement. Specifically, a Landmark-Guided Adaptive Encoder (LGAE) calibrates regional appearance features with landmark geometry through Bi-branch Gated Cross Attention (BGCA), suppressing noisy responses and producing expression-relevant representations. In parallel, a Vision-Language Enhancement Strategy (VLES) refines the generalizable visual features from a frozen CLIP image encoder into expression-specific visual representations. Based on them, an Expression-Conditioned Prompting (ECP) mechanism adapts fixed class-level textual prompts from the frozen CLIP text encoder into instance-aware textual representations. The resulting visual-textual representations are aligned as adaptive semantic priors to enhance robustness and generalization. Quantitative and qualitative experiments show that LaCoVL-FER outperforms state-of-the-art methods on RAF-DB, FERPlus, and AffectNet. The code is available at https://github.com/ylin06804/LaCoVL-FER.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。