arXiv:2410.22506cs.CV2024-10被引 24

用软标签提升面部表情识别,更贴近真实混合情绪。

AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels

  • 采用多情绪共存的软标签,每个情绪带置信度。
  • 构建新数据集AffectNet+,含年龄、性别等10+类元信息。
  • 适合情绪识别、人机交互研究者使用,缓解数据偏差。

自动面部表情识别(FER)因类内差异和类间相似性而困难,尤其在混合情绪(复合表情)场景下。现有数据集如AffectNet仅提供单一情绪标签(硬标签)。为缓解此类挑战并提供更真实的情绪描述,本文提出一种新标注方法:图像可标记多个情绪类别(软标签),每个类别附带置信度。我们引入软标签概念,提出一种新方法精确计算软标签——即表示单张人脸中多个情绪同时存在的向量。该方法有助于平滑分类边界、支持多标签标注,并减轻数据偏倚与不平衡问题。基于AffectNet,我们构建下一代数据集AffectNet+,包含软标签、三类复杂度子集及年龄、性别、种族、头姿、面部关键点、愉悦度与唤醒度等附加元数据。AffectNet+将对研究者公开。

原文摘要 · Abstract (English)

Automated Facial Expression Recognition (FER) is challenging due to intra-class variations and inter-class similarities. FER can be especially difficult when facial expressions reflect a mixture of various emotions (aka compound expressions). Existing FER datasets, such as AffectNet, provide discrete emotion labels (hard-labels), where a single category of emotion is assigned to an expression. To alleviate inter- and intra-class challenges, as well as provide a better facial expression descriptor, we propose a new approach to create FER datasets through a labeling method in which an image is labeled with more than one emotion (called soft-labels), each with different confidences. Specifically, we introduce the notion of soft-labels for facial expression datasets, a new approach to affective computing for more realistic recognition of facial expressions. To achieve this goal, we propose a novel methodology to accurately calculate soft-labels: a vector representing the extent to which multiple categories of emotion are simultaneously present within a single facial expression. Finding smoother decision boundaries, enabling multi-labeling, and mitigating bias and imbalanced data are some of the advantages of our proposed method. Building upon AffectNet, we introduce AffectNet+, the next-generation facial expression dataset. This dataset contains soft-labels, three categories of data complexity subsets, and additional metadata such as age, gender, ethnicity, head pose, facial landmarks, valence, and arousal. AffectNet+ will be made publicly accessible to researchers.

表情识别软标签数据集情感计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。