arXiv:2412.02530cs.CVcs.AI2024-12被引 1

用小波变换增强面部表情编辑,更好保留身份特征。

WEM-GAN: Wavelet transform based facial expression manipulation

  • 结合小波变换与U-net生成器,提升细节保留能力。
  • 引入高频判别器和对抗损失,生成更丰富的面部细节。
  • 在AffectNet数据集上表现优异,适合身份敏感的表情编辑场景。

面部表情操控旨在改变人脸表情而不影响身份识别。以往方法依赖表情标签引导,但常导致面部特征细节丢失,削弱身份信息。本文提出WEM-GAN(基于小波变换的面部表情操控生成对抗网络),通过小波变换与U-net自编码器结构结合,增强生成器对原始图像细节的保留能力;同时设计高频分量判别器,并采用高频域对抗损失,进一步约束模型优化,使生成图像具备更丰富的细节。此外,利用编码器与解码器间的残差连接,以及多次使用相对动作单元(AUs),缩小生成表情与目标表情的差距。大量定性与定量实验表明,该模型在AffectNet数据集上显著提升身份特征保持、编辑能力与图像质量,在平均内容距离(ACD)和表情距离(ED)等指标上表现更优。

原文摘要 · Abstract (English)

Facial expression manipulation aims to change human facial expressions without affecting face recognition. In order to transform the facial expressions to target expressions, previous methods relied on expression labels to guide the manipulation process. However, these methods failed to preserve the details of facial features, which causes the weakening or the loss of identity information in the output image. In our work, we propose WEM-GAN, in short for wavelet-based expression manipulation GAN, which puts more efforts on preserving the details of the original image in the editing process. Firstly, we take advantage of the wavelet transform technique and combine it with our generator with a U-net autoencoder backbone, in order to improve the generator's ability to preserve more details of facial features. Secondly, we also implement the high-frequency component discriminator, and use high-frequency domain adversarial loss to further constrain the optimization of our model, providing the generated face image with more abundant details. Additionally, in order to narrow the gap between generated facial expressions and target expressions, we use residual connections between encoder and decoder, while also using relative action units (AUs) several times. Extensive qualitative and quantitative experiments have demonstrated that our model performs better in preserving identity features, editing capability, and image generation quality on the AffectNet dataset. It also shows superior performance in metrics such as Average Content Distance (ACD) and Expression Distance (ED).

表情编辑小波变换生成对抗网络身份保留

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。