arXiv:2412.06295cs.CV2024-12CVPR被引 6

通过动态调整训练步数,让模型在不同噪声阶段学习更均衡,提升生成质量。

See Further When Clear: Curriculum Consistency Model

  • 按噪声强度动态调节教师模型迭代步数,平衡各阶段学习难度。
  • 在CIFAR-10上实现1.64的FID,在ImageNet 64x64上达2.18,单步采样表现优。
  • 适用于扩散与流匹配模型,尤其提升大模型图文对齐与结构准确性。

扩散模型和流匹配模型的采样效率近年大幅提升,主要依赖一致性蒸馏(CD),即让学生模型模仿教师模型在较晚时间步的输出。然而我们发现,学生模型在不同时间步上的学习复杂度差异显著,导致CD性能不佳。为此,我们提出课程一致性模型(CCM),通过将每个时间步的蒸馏过程视为一个课程,引入基于峰值信噪比(PSNR)的学习复杂度度量,并在噪声强度较低时让教师模型多迭代几步,以保持各时间步学习复杂度一致。该方法在CIFAR-10上实现1.64的单步采样FID,在ImageNet 64x64上达2.18。进一步扩展至大规模文本到图像模型,验证其在扩散模型(Stable Diffusion XL)和流匹配模型(Stable Diffusion 3)上均具良好泛化性。生成样本显示更强的图文对齐与语义结构,因CCM在大时间步扩大蒸馏步数,有效减少累积误差。

原文摘要 · Abstract (English)

Significant advances have been made in the sampling efficiency of diffusion models and flow matching models, driven by Consistency Distillation (CD), which trains a student model to mimic the output of a teacher model at a later timestep. However, we found that the learning complexity of the student model varies significantly across different timesteps, leading to suboptimal performance in CD.To address this issue, we propose the Curriculum Consistency Model (CCM), which stabilizes and balances the learning complexity across timesteps. Specifically, we regard the distillation process at each timestep as a curriculum and introduce a metric based on Peak Signal-to-Noise Ratio (PSNR) to quantify the learning complexity of this curriculum, then ensure that the curriculum maintains consistent learning complexity across different timesteps by having the teacher model iterate more steps when the noise intensity is low. Our method achieves competitive single-step sampling Fréchet Inception Distance (FID) scores of 1.64 on CIFAR-10 and 2.18 on ImageNet 64x64.Moreover, we have extended our method to large-scale text-to-image models and confirmed that it generalizes well to both diffusion models (Stable Diffusion XL) and flow matching models (Stable Diffusion 3). The generated samples demonstrate improved image-text alignment and semantic structure, since CCM enlarges the distillation step at large timesteps and reduces the accumulated error.

一致性蒸馏采样效率图像生成课程学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。