用少量样本让文生图模型精准生成特定领域图像
DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning
- 按属性驱动微调，解决少样本领域生成难题
- 在多个新领域上生成质量优于现有方法
- 适合需要快速适配新视觉风格的研究者
尽管大规模预训练的文生图模型已能根据文本提示生成多样图像，但在生成特定领域图像时仍受限，尤其当目标领域难以描述或模型从未见过。本文提出DomainGallery，一种少样本域驱动图像生成方法，通过属性中心化的微调策略，在少量目标数据上优化Stable Diffusion模型。该方法包含先验属性擦除、属性解耦、正则化与增强等技术，专门针对少样本域生成中的关键挑战。大量实验验证了DomainGallery在多种域驱动生成场景下的优越性能。代码已开源。
原文摘要 · Abstract (English)
The recent progress in text-to-image models pretrained on large-scale datasets has enabled us to generate various images as long as we provide a text prompt describing what we want. Nevertheless, the availability of these models is still limited when we expect to generate images that fall into a specific domain either hard to describe or just unseen to the models. In this work, we propose DomainGallery, a few-shot domain-driven image generation method which aims at finetuning pretrained Stable Diffusion on few-shot target datasets in an attribute-centric manner. Specifically, DomainGallery features prior attribute erasure, attribute disentanglement, regularization and enhancement. These techniques are tailored to few-shot domain-driven generation in order to solve key issues that previous works have failed to settle. Extensive experiments are given to validate the superior performance of DomainGallery on a variety of domain-driven generation scenarios. Codes are available at https://github.com/Ldhlwh/DomainGallery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。