用深度学习自动发现可解释的面部表情编码,比传统方法更精准。
Discrete Facial Encoding: : A Framework for Data-driven Facial Display Discovery
- 基于3D人脸模型提取去身份化表情特征,再用向量量化生成离散表情代码。
- 在压力、人格、抑郁识别任务中表现优于FACS和主流视觉模型。
- 代码可复用,适合心理学与情感计算研究者使用。
面部表情分析对理解人类行为至关重要,但现有编码系统如面部动作编码系统(FACS)受限于覆盖范围有限且需昂贵的人工标注。本文提出离散面部编码(DFE),一种无监督、数据驱动的紧凑可解释面部表情字典,通过残差向量量化变分自编码器(RVQ-VAE)从3D网格序列中学习。首先利用3D可变形模型(3DMM)从图像中提取与身份无关的表情特征,有效解耦头姿和面部几何等因素;随后使用RVQ-VAE对这些特征进行编码,生成共享码本中的离散标记序列,每个标记代表一个特定且可复用的面部形变模式。大量实验表明,DFE捕捉的面部行为比FACS及其他编码方式更精确。我们在三个高层心理任务中评估该表示:压力检测、人格预测和抑郁检测。基于学习到的标记构建的简单词袋模型,持续优于基于FACS的流程及强大的图像与视频表征学习模型(如掩码自编码器)。进一步分析显示,该表示涵盖更多样化的面部表现,凸显其作为心理与情感计算中可扩展、高效的FACS替代方案潜力。
原文摘要 · Abstract (English)
Facial expression analysis is central to understanding human behavior, yet existing coding systems such as the Facial Action Coding System (FACS) are constrained by limited coverage and costly manual annotation. In this work, we introduce Discrete Facial Encoding (DFE), an unsupervised, data-driven alternative of compact and interpretable dictionary of facial expressions from 3D mesh sequences learned through a Residual Vector Quantized Variational Autoencoder (RVQ-VAE). Our approach first extracts identity-invariant expression features from images using a 3D Morphable Model (3DMM), effectively disentangling factors such as head pose and facial geometry. We then encode these features using an RVQ-VAE, producing a sequence of discrete tokens from a shared codebook, where each token captures a specific, reusable facial deformation pattern that contributes to the overall expression. Through extensive experiments, we demonstrate that Discrete Facial Encoding captures more precise facial behaviors than FACS and other facial encoding alternatives. We evaluate the utility of our representation across three high-level psychological tasks: stress detection, personality prediction, and depression detection. Using a simple Bag-of-Words model built on top of the learned tokens, our system consistently outperforms both FACS-based pipelines and strong image and video representation learning models such as Masked Autoencoders. Further analysis reveals that our representation covers a wider variety of facial displays, highlighting its potential as a scalable and effective alternative to FACS for psychological and affective computing applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。