用文本和表情动作单元生成逼真人脸表情图像,提升表情识别性能。
SynFER: Towards Boosting Facial Expression Recognition with Synthetic Data
- 基于文本描述和面部动作单元控制生成合成表情图像。
- 仅用合成数据在AffectNet上达到67.23%准确率,规模扩大5倍后达69.84%。
- 适合缺乏标注数据但需高效训练表情识别模型的研究者。
人脸表情数据集因标注主观性和采集耗时而规模有限,制约了深度学习模型尤其是基础模型的性能。为应对这一挑战,我们提出SynFER(基于精细控制的面部表情合成),通过高阶文本描述及更细粒度的面部动作单元(Facial Action Units)生成合成表情图像。为保障生成数据质量,引入语义引导机制以控制生成过程,并设计伪标签生成器修正合成图像的表情标签。在表示学习上,结合真实与合成数据进行大量实验,验证了方法有效性。结果显示,仅使用与AffectNet训练集等量的合成数据即可实现67.23%的分类准确率;当数据规模扩大至五倍时,准确率提升至69.84%。代码已公开。
原文摘要 · Abstract (English)
Facial expression datasets remain limited in scale due to the subjectivity of annotations and the labor-intensive nature of data collection. This limitation poses a significant challenge for developing modern deep learning-based facial expression analysis models, particularly foundation models, that rely on large-scale data for optimal performance. To tackle the overarching and complex challenge, instead of introducing a new large-scale dataset, we introduce SynFER (Synthesis of Facial Expressions with Refined Control), a novel synthetic framework for synthesizing facial expression image data based on high-level textual descriptions as well as more fine-grained and precise control through facial action units. To ensure the quality and reliability of the synthetic data, we propose a semantic guidance technique to steer the generation process and a pseudo-label generator to help rectify the facial expression labels for the synthetic images. To demonstrate the generation fidelity and the effectiveness of the synthetic data from SynFER, we conduct extensive experiments on representation learning using both synthetic data and real-world data. Results validate the efficacy of our approach and the synthetic data. Notably, our approach achieves a 67.23% classification accuracy on AffectNet when training solely with synthetic data equivalent to the AffectNet training set size, which increases to 69.84% when scaling up to five times the original size. Code is available here.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。