用领域引导提升扩散模型迁移效果，无需重新训练。
Domain Guidance: A Simple Transfer Approach for a Pre-trained Diffusion Model
- 通过预训练知识引导采样过程，实现领域对齐。
- 在多个基准上FID提升19.6%，FD$_\text{DINOv2}$提升23.4%。
- 兼容已有微调模型，即插即用无须额外训练。
扩散模型的最新进展推动了生成建模的飞跃，但其逼真输出常伴随模型规模膨胀与计算成本上升。因此，基于现成模型构建个性化扩散模型成为可行替代方案。本文提出一种新视角下的条件生成迁移方法——领域引导（Domain Guidance），利用预训练知识指导采样过程，使其聚焦目标领域。该方法形式上类似先进的无分类器引导，能更好实现领域对齐并生成更高质量结果。我们从实证与理论两方面分析其机制。实验表明，该方法在多个迁移任务中表现卓越，相比标准微调，FID指标提升超过19.6%，FD$_\text{DINOv2}$提升23.4%。值得注意的是，现有微调模型可无缝集成领域引导，无需额外训练即可获得性能增益。
原文摘要 · Abstract (English)
Recent advancements in diffusion models have revolutionized generative modeling. However, the impressive and vivid outputs they produce often come at the cost of significant model scaling and increased computational demands. Consequently, building personalized diffusion models based on off-the-shelf models has emerged as an appealing alternative. In this paper, we introduce a novel perspective on conditional generation for transferring a pre-trained model. From this viewpoint, we propose *Domain Guidance*, a straightforward transfer approach that leverages pre-trained knowledge to guide the sampling process toward the target domain. Domain Guidance shares a formulation similar to advanced classifier-free guidance, facilitating better domain alignment and higher-quality generations. We provide both empirical and theoretical analyses of the mechanisms behind Domain Guidance. Our experimental results demonstrate its substantial effectiveness across various transfer benchmarks, achieving over a 19.6% improvement in FID and a 23.4% improvement in FD$_\text{DINOv2}$ compared to standard fine-tuning. Notably, existing fine-tuned models can seamlessly integrate Domain Guidance to leverage these benefits, without additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。