通过可控人脸编辑增强稀缺标注数据,提升表情分析模型性能。
Controlled Face Manipulation and Synthesis for Data Augmentation
- 在预训练生成器的语义潜空间中,用轻量线性模型实现精准动作单元编辑。
- 生成数据使表情检测准确率提升,且减少属性纠缠与共激活偏差。
- 适合需要数据增强的面部表情分析、少样本学习场景,尤其关注身份保真度。
深度学习视觉模型依赖充足标注数据,但许多应用面临标签稀缺与类别不平衡问题。可控图像编辑可扩充稀缺标签数据,但现有方法常引入伪影并混淆非目标属性。本文聚焦面部表情分析中的动作单元(AU)操纵任务,其标注成本高且存在多属性共激活导致的纠缠。提出一种基于预训练人脸生成器(扩散自编码器)语义潜空间的编辑方法:通过(i)考虑AU共激活的依赖感知条件控制,(ii)正交投影消除无关属性方向(如眼镜),结合表情中性化步骤,实现绝对AU编辑。利用生成数据平衡各类别出现频率,并通过受控合成丰富身份与人口统计多样性。实验显示,使用生成数据训练的AU检测器准确率更高,预测更解耦,且避免共激活捷径,性能优于其他数据高效训练策略;学习曲线分析表明效果相当于增加大量真实标注。相比先前方法,本方法编辑更强、伪影更少、身份保持更好。
原文摘要 · Abstract (English)
Deep learning vision models excel with abundant supervision, but many applications face label scarcity and class imbalance. Controllable image editing can augment scarce labeled data, yet edits often introduce artifacts and entangle non-target attributes. We study this in facial expression analysis, targeting Action Unit (AU) manipulation where annotation is costly and AU co-activation drives entanglement. We present a facial manipulation method that operates in the semantic latent space of a pre-trained face generator (Diffusion Autoencoder). Using lightweight linear models, we reduce entanglement of semantic features via (i) dependency-aware conditioning that accounts for AU co-activation, and (ii) orthogonal projection that removes nuisance attribute directions (e.g., glasses), together with an expression neutralization step to enable absolute AU edit. We use these edits to balance AU occurrence by editing labeled faces and to diversify identities/demographics via controlled synthesis. Augmenting AU detector training with the generated data improves accuracy and yields more disentangled predictions with fewer co-activation shortcuts, outperforming alternative data-efficient training strategies and suggesting improvements similar to what would require substantially more labeled data in our learning-curve analysis. Compared to prior methods, our edits are stronger, produce fewer artifacts, and preserve identity better.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。