统一蒸馏与适配,用单阶段方法实现少步少样本高质量图像生成。
Uni-DAD: Unified Distillation and Adaptation of Diffusion Models for Few-step Few-shot Image Generation
- 单阶段联合蒸馏与适配,避免两阶段流程复杂性。
- 仅需少于4步采样即达当前最优质量,且多样性更优。
- 适合少样本图像生成与个性化定制场景,尤其结构差异大的领域。
扩散模型(DMs)生成高质量图像,但适应新领域时采样成本高。蒸馏后的DMs虽更快,但通常局限于教师模型的领域。现有方法依赖两阶段流程:先适配再蒸馏或反之,增加设计复杂度并常导致质量下降。本文提出Uni-DAD,一种单阶段统一蒸馏与适配框架。其结合双域分布匹配蒸馏(DMD)目标,引导学生模型逼近源教师和目标教师的分布;同时引入多头生成对抗网络(GAN)损失,在多个特征尺度上增强目标真实感。源域蒸馏保留多样源知识,多头GAN提升训练稳定性、减少过拟合,尤其在少样本情况下表现优异。引入目标教师有助于适应结构差异较大的领域。在两个全面的少样本图像生成(FSIG)和主题驱动个性化(SDP)基准上评估,使用扩散主干网络,Uni-DAD在少于4步采样下仍达到或超越当前最优适配方法的质量,且多数情况优于两阶段流水线,在质量和多样性上均有提升。代码见:https://github.com/yaramohamadi/uni-DAD。
原文摘要 · Abstract (English)
Diffusion models (DMs) produce high-quality images, yet their sampling remains costly when adapted to new domains. Distilled DMs are faster but typically remain confined within their teacher's domain. Thus, fast and high-quality generation for novel domains relies on two-stage pipelines: Adapt-then-Distill or Distill-then-Adapt. However, both add design complexity and often degrade quality or diversity. We introduce Uni-DAD, a single-stage pipeline that unifies DM distillation and adaptation. It couples two training signals: (i) a dual-domain distribution-matching distillation (DMD) objective that guides the student toward the distributions of the source teacher and a target teacher, and (ii) a multi-head generative adversarial network (GAN) loss that encourages target realism across multiple feature scales. The source domain distillation preserves diverse source knowledge, while the multi-head GAN stabilizes training and reduces overfitting, especially in few-shot regimes. The inclusion of a target teacher facilitates adaptation to more structurally distant domains. We evaluate Uni-DAD on two comprehensive benchmarks for few-shot image generation (FSIG) and subject-driven personalization (SDP) using diffusion backbones. It delivers better or comparable quality to state-of-the-art (SoTA) adaptation methods even with less than 4 sampling steps, and often surpasses two-stage pipelines in quality and diversity. Code: https://github.com/yaramohamadi/uni-DAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。