一个模型搞定所有美妆任务，还能用文字控制试妆效果。
MAD: Makeup All-in-One with Cross-Domain Diffusion Model
- 用跨域扩散模型统一处理美妆各类任务
- 仅通过更换嵌入向量实现不同任务无缝切换
- 新增文本标注数据集，支持文字驱动试妆
现有美妆技术通常需为不同输入设计多个模型，且在不同任务间对齐特征，导致复杂度高。此外，缺乏文本引导的试妆功能，用户体验不佳。本文首次提出使用单一模型完成多种美妆任务。具体而言，将各类美妆任务建模为跨域翻译问题，利用跨域扩散模型实现统一处理。不同于以往依赖独立编码器-解码器结构或循环机制的方法，我们采用不同领域嵌入实现领域控制，仅通过替换嵌入即可实现任务切换，无需额外模块。此外，为支持精准的文生美妆应用，我们基于MT数据集扩展构建了MT-Text数据集，提升了美妆技术的实用性。
原文摘要 · Abstract (English)
Existing makeup techniques often require designing multiple models to handle different inputs and align features across domains for different makeup tasks, e.g., beauty filter, makeup transfer, and makeup removal, leading to increased complexity. Another limitation is the absence of text-guided makeup try-on, which is more user-friendly without needing reference images. In this study, we make the first attempt to use a single model for various makeup tasks. Specifically, we formulate different makeup tasks as cross-domain translations and leverage a cross-domain diffusion model to accomplish all tasks. Unlike existing methods that rely on separate encoder-decoder configurations or cycle-based mechanisms, we propose using different domain embeddings to facilitate domain control. This allows for seamless domain switching by merely changing embeddings with a single model, thereby reducing the reliance on additional modules for different tasks. Moreover, to support precise text-to-makeup applications, we introduce the MT-Text dataset by extending the MT dataset with textual annotations, advancing the practicality of makeup technologies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。