用自身内部动态指导扩散模型,提升生成质量与训练效率。
Guiding a Diffusion Transformer with the Internal Dynamics of Itself
- 训练时引入中间层辅助监督,采样时外推深层输出
- ImageNet上FID低至1.34,结合CFG达1.19新纪录
- 无需额外训练或复杂退化策略,适合追求高质生成的开发者
扩散模型能有效捕捉数据整体分布,但因训练数据不足,难以覆盖低概率区域,导致生成图像质量下降。传统分类器无关引导(CFG)虽可引导样本进入高概率区域,却常引发样本简化或失真。另一种通过劣化版本引导的方法受限于复杂的退化设计、额外训练和采样步骤。本文提出简单有效的内部引导(IG)策略:在训练中对中间层施加辅助监督,并在采样时外推中间层与深层输出以生成结果。该方法显著提升训练效率与生成质量。在ImageNet 256x256上,SiT-XL/2+IG在80和800轮时分别达到FID=5.31和1.75;更令人瞩目的是,LightningDiT-XL/1+IG实现FID=1.34,大幅领先其他方法。结合CFG后,LightningDiT-XL/1+IG达到当前最优的FID=1.19。
原文摘要 · Abstract (English)
The diffusion model presents a powerful ability to capture the entire (conditional) data distribution. However, due to the lack of sufficient training and data to learn to cover low-probability areas, the model will be penalized for failing to generate high-quality images corresponding to these areas. To achieve better generation quality, guidance strategies such as classifier free guidance (CFG) can guide the samples to the high-probability areas during the sampling stage. However, the standard CFG often leads to over-simplified or distorted samples. On the other hand, the alternative line of guiding diffusion model with its bad version is limited by carefully designed degradation strategies, extra training and additional sampling steps. In this paper, we proposed a simple yet effective strategy Internal Guidance (IG), which introduces an auxiliary supervision on the intermediate layer during training process and extrapolates the intermediate and deep layer's outputs to obtain generative results during sampling process. This simple strategy yields significant improvements in both training efficiency and generation quality on various baselines. On ImageNet 256x256, SiT-XL/2+IG achieves FID=5.31 and FID=1.75 at 80 and 800 epochs. More impressively, LightningDiT-XL/1+IG achieves FID=1.34 which achieves a large margin between all of these methods. Combined with CFG, LightningDiT-XL/1+IG achieves the current state-of-the-art FID of 1.19.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。