用文本生成带精确表情标注的逼真人脸，解决数据不足问题
MAUGen: A Unified Diffusion Approach for Multi-Identity Facial Expression and AU Label Generation
- 基于扩散模型联合生成人脸图像与动作单元标签
- 可生成跨身份、多表情的逼真图像与强度标注
- 适合需要多样化带标注人脸数据的研究者
缺乏大规模、人口统计多样且带有精确动作单元（AU）出现与强度标注的人脸图像，长期制约着通用AU识别系统的发展。本文提出MAUGen，一种基于扩散的多模态框架，可仅通过一个文本描述生成大量逼真人脸表情及解剖学一致的AU标签（包括出现与强度），并支持多种身份。MAUGen包含两个核心模块：(1) 多模态表示学习（MRL）模块，在统一潜在空间中捕捉文本描述、人脸身份、表情图像与AU激活之间的关系；(2) 扩散式图像标签生成器（DIG），将联合表示解码为跨身份对齐的人脸图像-标签对。在此框架下，我们构建了大规模多模态合成数据集Multi-Identity Facial Action（MIFA），涵盖全面的AU标注与身份变化。大量实验表明，MAUGen在生成逼真、人口多样性丰富的人脸图像及语义对齐的AU标签方面优于现有方法。
原文摘要 · Abstract (English)
The lack of large-scale, demographically diverse face images with precise Action Unit (AU) occurrence and intensity annotations has long been recognized as a fundamental bottleneck in developing generalizable AU recognition systems. In this paper, we propose MAUGen, a diffusion-based multi-modal framework that jointly generates a large collection of photorealistic facial expressions and anatomically consistent AU labels, including both occurrence and intensity, conditioned on a single descriptive text prompt. Our MAUGen involves two key modules: (1) a Multi-modal Representation Learning (MRL) module that captures the relationships among the paired textual description, facial identity, expression image, and AU activations within a unified latent space; and (2) a Diffusion-based Image label Generator (DIG) that decodes the joint representation into aligned facial image-label pairs across diverse identities. Under this framework, we introduce Multi-Identity Facial Action (MIFA), a large-scale multimodal synthetic dataset featuring comprehensive AU annotations and identity variations. Extensive experiments demonstrate that MAUGen outperforms existing methods in synthesizing photorealistic, demographically diverse facial images along with semantically aligned AU labels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。