用扩散模型实现无需训练的精准人脸化妆定制，支持多种条件控制。
DreamMakeup: Face Makeup Customization using Latent Diffusion Models
- 基于扩散模型的无训练框架，通过早期停止DDIM反演保持面部结构
- 支持参考图、颜色、文字描述等多种条件输入，实现高精度妆容定制
- 相比生成对抗网络更稳定，兼容大语言模型，计算成本低
全球美妆市场快速增长，虚拟试妆技术也随之进步。尽管生成对抗网络（GAN）已广泛应用，但仍面临训练不稳定和定制能力有限的问题。为此，我们提出DreamMakeup——一种基于扩散模型的无训练妆容定制方法，利用扩散模型在可控性和真实图像编辑上的优势。该方法采用早期停止的DDIM反演，保留人脸结构与身份特征，同时通过参考图像、特定RGB颜色或文本描述等条件输入实现广泛定制。实验表明，相较于现有基于GAN和近期扩散模型的方法，DreamMakeup在定制能力、色彩匹配度、身份保持及文本描述兼容性方面均有显著提升，且计算开销较低。
原文摘要 · Abstract (English)
The exponential growth of the global makeup market has paralleled advancements in virtual makeup simulation technology. Despite the progress led by GANs, their application still encounters significant challenges, including training instability and limited customization capabilities. Addressing these challenges, we introduce DreamMakup - a novel training-free Diffusion model based Makeup Customization method, leveraging the inherent advantages of diffusion models for superior controllability and precise real-image editing. DreamMakeup employs early-stopped DDIM inversion to preserve the facial structure and identity while enabling extensive customization through various conditioning inputs such as reference images, specific RGB colors, and textual descriptions. Our model demonstrates notable improvements over existing GAN-based and recent diffusion-based frameworks - improved customization, color-matching capabilities, identity preservation and compatibility with textual descriptions or LLMs with affordable computational costs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。