揭示扩散模型生成图像时语义信息出现的精确时间点。
Measuring Semantic Information Production in Generative Diffusion Models
- 用贝叶斯分类器估计噪声状态下的类别条件熵
- 发现语义信息传递在中间阶段最强，末期趋近于零
- 不同类别决策时间不同，揭示生成过程的非均质性
众所周知，扩散模型生成图像时，语义与结构特征会在反向过程中不同时段涌现，这一现象与磁性材料等物理相变相关。本文提出一种通用的信息论方法，用于测量生成过程中类别语义“决策”发生的时间。通过在线贝叶斯分类器公式，估算给定噪声状态下的类别条件熵，并利用其时间导数识别信息传递最高的时段。我们在一维高斯混合模型和在CIFAR10上训练的DDPM模型上验证了该方法。结果表明，语义信息传递主要集中在扩散的中间阶段，而在最终阶段趋于消失。但不同类别的熵率曲线存在显著差异，说明各类别“语义决策”发生在不同的中间时间点。
原文摘要 · Abstract (English)
It is well known that semantic and structural features of the generated images emerge at different times during the reverse dynamics of diffusion, a phenomenon that has been connected to physical phase transitions in magnets and other materials. In this paper, we introduce a general information-theoretic approach to measure when these class-semantic "decisions" are made during the generative process. By using an online formula for the optimal Bayesian classifier, we estimate the conditional entropy of the class label given the noisy state. We then determine the time intervals corresponding to the highest information transfer between noisy states and class labels using the time derivative of the conditional entropy. We demonstrate our method on one-dimensional Gaussian mixture models and on DDPM models trained on the CIFAR10 dataset. As expected, we find that the semantic information transfer is highest in the intermediate stages of diffusion while vanishing during the final stages. However, we found sizable differences between the entropy rate profiles of different classes, suggesting that different "semantic decisions" are located at different intermediate times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。