无需配对数据，用扩散模型实现高保真分层妆容迁移
SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models
- 基于潜空间扩散模型，自监督解耦重建妆容特征
- 通过拉普拉斯金字塔分解纹理，支持多样妆容风格适配
- 动态对齐模块纠正内容与妆容间的域差异，提升迁移自然度
本文研究了极具挑战性的妆容迁移任务，旨在将多种妆容风格精确且自然地应用到目标人脸图像上。由于缺乏成对数据，现有方法通常依赖合成的伪真实样本进行训练，导致妆容保真度较低。此外，不同妆容风格对人脸的影响各异，但现有方法难以处理这种多样性。为此，我们提出一种基于潜空间扩散模型的自监督分层妆容迁移（SHMT）方法。遵循“解耦-重建”范式，该方法在自监督框架下运行，避免了不准确伪配对数据的误导。为进一步适应多样妆容风格，通过拉普拉斯金字塔分解层级纹理细节，并选择性地注入内容表征。最后，设计了一种新型迭代双对齐（IDA）模块，动态调整扩散模型的注入条件，有效纠正内容与妆容表征间因域差异带来的对齐误差。大量定量与定性分析验证了方法的有效性。代码已开源。
原文摘要 · Abstract (English)
This paper studies the challenging task of makeup transfer, which aims to apply diverse makeup styles precisely and naturally to a given facial image. Due to the absence of paired data, current methods typically synthesize sub-optimal pseudo ground truths to guide the model training, resulting in low makeup fidelity. Additionally, different makeup styles generally have varying effects on the person face, but existing methods struggle to deal with this diversity. To address these issues, we propose a novel Self-supervised Hierarchical Makeup Transfer (SHMT) method via latent diffusion models. Following a "decoupling-and-reconstruction" paradigm, SHMT works in a self-supervised manner, freeing itself from the misguidance of imprecise pseudo-paired data. Furthermore, to accommodate a variety of makeup styles, hierarchical texture details are decomposed via a Laplacian pyramid and selectively introduced to the content representation. Finally, we design a novel Iterative Dual Alignment (IDA) module that dynamically adjusts the injection condition of the diffusion model, allowing the alignment errors caused by the domain gap between content and makeup representations to be corrected. Extensive quantitative and qualitative analyses demonstrate the effectiveness of our method. Our code is available at \url{https://github.com/Snowfallingplum/SHMT}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。