让深度模型学会像专家一样看脸,自动定位表情关键区域。
Spatial Action Unit Cues for Interpretable Deep Facial Expression Recognition
- 用面部动作单元热图引导训练,让模型关注表情关键区域。
- 在RAF-DB和AffectNet上保持高准确率,同时提升可解释性。
- 无需额外标注,适配各种CNN/Transformer模型,通用性强。
尽管当前最先进的面部表情识别(FER)分类器能达到很高准确率,但缺乏可解释性,这对终端用户很重要。专家通常通过动作单元(AUs)代码本关联面部区域来解释表情。本文沿用这一专家思路,提出一种新学习策略,将AU线索显式融入分类器训练中,实现深度可解释模型。训练时,利用输入图像的表情标签、面部关键点及AU代码本构建表示表情判别区域的AU热图,该空间线索用于约束分类器各层特征与热图相关。通过组合损失函数,模型在正确分类的同时生成与AU图对齐的逐层视觉注意力,模拟专家决策过程。该策略仅依赖图像类别标签监督,无需额外人工标注。方法具有通用性,可无缝应用于任何CNN或Transformer架构,无需结构修改或显著增加训练时间。在RAF-DB和AffectNet两个公开数据集上的广泛评估表明,该方法在不降低分类性能的前提下显著提升层间可解释性。此外,我们还探索了基于类激活图(CAM)的可解释分类器,结果表明本方法同样能增强CAM的可解释性。
原文摘要 · Abstract (English)
Although state-of-the-art classifiers for facial expression recognition (FER) can achieve a high level of accuracy, they lack interpretability, an important feature for end-users. Experts typically associate spatial action units (AUs) from a codebook to facial regions for the visual interpretation of expressions. In this paper, the same expert steps are followed. A new learning strategy is proposed to explicitly incorporate AU cues into classifier training, allowing to train deep interpretable models. During training, this AU codebook is used, along with the input image expression label, and facial landmarks, to construct a AU heatmap that indicates the most discriminative image regions of interest w.r.t the facial expression. This valuable spatial cue is leveraged to train a deep interpretable classifier for FER. This is achieved by constraining the spatial layer features of a classifier to be correlated with AU heatmaps. Using a composite loss, the classifier is trained to correctly classify an image while yielding interpretable visual layer-wise attention correlated with AU maps, simulating the expert decision process. Our strategy only relies on image class expression for supervision, without additional manual annotations. Our new strategy is generic, and can be applied to any deep CNN- or transformer-based classifier without requiring any architectural change or significant additional training time. Our extensive evaluation on two public benchmarks RAF-DB, and AffectNet datasets shows that our proposed strategy can improve layer-wise interpretability without degrading classification performance. In addition, we explore a common type of interpretable classifiers that rely on class activation mapping (CAM) methods, and show that our approach can also improve CAM interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。