用合成数据缓解表情识别中的数据不平衡与隐私问题。
On Applicability of Synthetic Datasets for Facial Expression Recognition
- 通过教师模型伪标签+扩散模型生成+任务感知GAN编辑构建隐私保护数据集。
- 合成数据在跨数据集测试中表现接近真实数据，有效缓解类别不平衡。
- 适合关注数据隐私、小样本训练或需增强数据多样性的研究者。
面部表情识别面临两大核心挑战：公开数据集存在类别不平衡，影响模型泛化能力；同时受隐私和数据采集限制，难以构建大规模平衡数据集。为应对这些问题，本文提出三种互补策略构建隐私保护的面部表情识别数据集：(i) 使用置信度阈值的教师模型对大规模无标签人脸集合进行伪标注；(ii) 基于扩散模型，以人口统计属性为提示生成表情图像；(iii) 基于任务感知的GAN实现表情修改，保持身份一致性和真实感。实验采用AffectNet、RAF-DB和FER2013等标准数据集，以DigiFace、DCFace、EmoNet-Face BIG作为伪标签来源，FFHQ用于生成合成图像。主实验使用经典CNN骨干网络IR50，也测试了更复杂的POSTERv1架构。通过跨数据集评估分析各策略的权衡关系。结果表明，合成数据可有效替代或补充真实数据，缓解类别不平衡与隐私约束。代码与生成数据集见：https://www.github.com/AliAZ98/SyntFER
原文摘要 · Abstract (English)
Facial Expression Recognition faces two core challenges. The first is class imbalance in public datasets, which skews the learning process and weakens generalization. The second is related to privacy and data collection constraints, which limit the sharing of facial images and restrict the creation of large, balanced datasets. To address these issues, we examine three complementary strategies for constructing privacy-preserving FER datasets in the standard seven discrete facial expression classes setting. Our strategies are: (i) pseudo-labeling large unlabeled face collections with a teacher model under a confidence-thresholding scheme, (ii) prompt-driven synthesis using diffusion models conditioned on demographic attributes, and (iii) task-aware GAN-based expression editing that modifies facial expression while preserving identity and realism. For training and evaluation, we employed widely adopted datasets, including AffectNet, RAF-DB, and FER2013. We utilized the synthetic datasets DigiFace, DCFace, and EmoNet-Face BIG as unlabeled sources for pseudo-labeling. Additionally, we utilized the FFHQ dataset as the source for generative synthesis. The main experiments are conducted using a classic CNN backbone, IR50, and we also explore a more complex architecture, POSTERv1, to assess its feasibility and robustness. Using cross-dataset evaluations, we analyze the trade-offs each strategy presents in curated datasets. The findings demonstrate how synthetic data can effectively substitute or be combined with real datasets to mitigate imbalance and privacy limitations. Code and generated datasets:https://www.github.com/AliAZ98/SyntFER
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。