arXiv:2507.15401cs.CV2025-07被引 5

提出新模型解决人脸表情识别中的遮挡问题,提升准确率。

Rethinking Occlusion in FER: A Semantic-Aware Perspective and Go Beyond

  • 用语义分割和关键点作为多模态先验,增强特征表达。
  • 在公开数据集和新构建的Occlu-FER上达到最先进性能。
  • 适合研究遮挡鲁棒性或实际部署中表情识别任务的人看。

由于普遍存在遮挡和数据集偏差,面部表情识别(FER)是一项具有挑战性的任务。当面部信息部分被遮挡时,现有模型难以提取有效特征,导致分类不准确。为此,我们提出ORSANet,主要贡献有三:第一,引入辅助多模态语义引导以消解遮挡歧义并学习高层语义知识,具体包括:1)使用语义分割图作为密集语义先验,生成语义增强的面部表征;2)使用面部关键点作为稀疏几何先验,缓解身份、性别等内在噪声。第二,设计多尺度交叉交互模块(MCM),自适应融合不同尺度下的关键点特征与语义增强表征。第三,提出动态对抗排斥增强损失(DARELoss),动态调整模糊类别的间隔,进一步提升模型区分相似表情的能力。我们还构建了首个面向遮挡的FER数据集Occlu-FER,用于在多种真实遮挡条件下进行专项鲁棒性分析。在公开基准和Occlu-FER上的大量实验表明,所提ORSANet取得当前最优识别性能。代码已公开于https://github.com/Wenyuzhy/ORSANet-master。

原文摘要 · Abstract (English)

Facial expression recognition (FER) is a challenging task due to pervasive occlusion and dataset biases. Especially when facial information is partially occluded, existing FER models struggle to extract effective facial features, leading to inaccurate classifications. In response, we present ORSANet, which introduces the following three key contributions: First, we introduce auxiliary multi-modal semantic guidance to disambiguate facial occlusion and learn high-level semantic knowledge, which is two-fold: 1) we introduce semantic segmentation maps as dense semantics prior to generate semantics-enhanced facial representations; 2) we introduce facial landmarks as sparse geometric prior to mitigate intrinsic noises in FER, such as identity and gender biases. Second, to facilitate the effective incorporation of these two multi-modal priors, we customize a Multi-scale Cross-interaction Module (MCM) to adaptively fuse the landmark feature and semantics-enhanced representations within different scales. Third, we design a Dynamic Adversarial Repulsion Enhancement Loss (DARELoss) that dynamically adjusts the margins of ambiguous classes, further enhancing the model's ability to distinguish similar expressions. We further construct the first occlusion-oriented FER dataset to facilitate specialized robustness analysis on various real-world occlusion conditions, dubbed Occlu-FER. Extensive experiments on both public benchmarks and Occlu-FER demonstrate that our proposed ORSANet achieves SOTA recognition performance. Code is publicly available at https://github.com/Wenyuzhy/ORSANet-master.

表情识别遮挡处理多模态融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。