针对低频类别生成质量差,提出按类别频率调整噪声调度。
Class-frequency Guided Noise Schedule for Diffusion Models
- 根据类别频率动态调整扩散过程中的噪声尺度,低频类用更大噪声。
- 在CIFAR-100-LT和ImageNet-LT上显著提升低频类样本质量与多样性。
- 适合处理数据不平衡场景下的图像生成与文本到图像生成任务。
本文首次研究扩散模型中类别频率与多尺度噪声调度之间的关系。对于基于分数的生成模型,低密度区域常导致分数估计不准确,影响生成质量。尽管多尺度噪声调度可在扩散过程中缓解此问题,但低频类别仍面临大范围低密度区域,导致分数估计误差大于高频类别。此外,高频类别倾向于主导分数空间,使多数数据点趋向生成高频类样本。因此,低频类生成样本质量差且多样性不足。为此,我们提出类别频率引导(CFRG)噪声调度,利用低频类别应使用更大尺度噪声的洞察。通过在图像生成、图像分类和文本到图像生成等任务上使用不平衡数据集(如CIFAR-100-LT和ImageNet-LT)进行实验,采用CFRG噪声调度后,相较基线方法取得显著提升,验证了频率统计在噪声调度设计中的关键作用。
原文摘要 · Abstract (English)
In this paper, we are the first to examine the correlations between class frequency and the multi-scale noise schedule within diffusion models. For score-based generative models, low-density regions often lead to inaccurately estimated scores, thereby compromising the generation quality. Although the multi-scale noise schedule can alleviate this issue during the diffusion process, low-frequency classes still face the challenge of large low-density regions, resulting in more inaccurate estimated scores than high-frequency classes. Furthermore, high-frequency classes tend to dominate the score space, causing a convergence of most data points towards generating samples from these classes. Consequently, samples generated within low-frequency classes exhibit suboptimal quality and limited diversity. To address this challenge, we propose the \textit{Class-frequency Guided (CFRG)} noise schedule, leveraging the insight that low-frequency classes should be endowed with larger-scale noises. To illustrate the effectiveness of our method, we conduct experiments on various tasks, including image generation, image classification, and text-to-image generation, using imbalanced datasets, \textit{i.e.}, CIFAR-100-LT, and ImageNet-LT. By employing the CFRG noise schedule, we achieve substantial improvements over baselines, manifesting the crucial role of frequency statistics in noise schedule design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。