将化妆特征解耦编码到高维空间,实现精准妆容迁移与生成
BeautyBank: Encoding Facial Makeup in Latent Space
- 通过解耦裸脸与妆容的局部模式特征,构建高维妆容编码空间
- 在32.4万对512x512图像上验证,显著提升妆容细节保留能力
- 适用于妆容生成、相似度度量等新场景,适合视觉编辑与数字人应用
妆容迁移、编辑与图像编码技术已展现出优异效果。然而,现有方法多聚焦于低维特征如颜色分布和图案,难以覆盖多样化妆容应用;而高维隐空间编码方法通常关注全局结构与风格,对妆容局部颜色与纹理细节捕捉不足。为此,我们提出BeautyBank,一种新型妆容编码器,可解耦裸脸与妆容的图案特征。该方法将妆容特征编码至高维空间,保留重建所需关键细节,并拓展了妆容研究的应用范围。我们还提出渐进式妆容调优(PMT)策略,强化细节保持并抑制无关属性。进一步探索了含妆容注入的面部图像生成与妆容相似度度量等新应用。大量实验证明,该方法具备更强任务适应性,在多个妆容相关领域具有广泛应用潜力。此外,为解决领域内缺乏大规模高质量成对数据的问题,我们构建了包含324,000对512×512像素图像的裸脸-妆容合成数据集(BMS)。
原文摘要 · Abstract (English)
The advancement of makeup transfer, editing, and image encoding has demonstrated their effectiveness and superior quality. However, existing makeup works primarily focus on low-dimensional features such as color distributions and patterns, limiting their versatillity across a wide range of makeup applications. Futhermore, existing high-dimensional latent encoding methods mainly target global features such as structure and style, and are less effective for tasks that require detailed attention to local color and pattern features of makeup. To overcome these limitations, we propose BeautyBank, a novel makeup encoder that disentangles pattern features of bare and makeup faces. Our method encodes makeup features into a high-dimensional space, preserving essential details necessary for makeup reconstruction and broadening the scope of potential makeup research applications. We also propose a Progressive Makeup Tuning (PMT) strategy, specifically designed to enhance the preservation of detailed makeup features while preventing the inclusion of irrelevant attributes. We further explore novel makeup applications, including facial image generation with makeup injection and makeup similarity measure. Extensive empirical experiments validate that our method offers superior task adaptability and holds significant potential for widespread application in various makeup-related fields. Furthermore, to address the lack of large-scale, high-quality paired makeup datasets in the field, we constructed the Bare-Makeup Synthesis Dataset (BMS), comprising 324,000 pairs of 512x512 pixel images of bare and makeup-enhanced faces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。