arXiv:2601.21200stat.MLcs.LG2026-01被引 1

通过控制交叉熵,首次理论证明了分类器引导扩散模型的有效性。

Provably Reliable Classifier Guidance via Cross-Entropy Control

  • 用交叉熵损失训练分类器,控制每步的误差
  • 分类器误差ε²对应引导向量误差为O(dε)
  • 为选择有效分类器提供理论依据,适合研究者参考

分类器引导的扩散模型通过在反向时间得分中加入分类器对数概率梯度来生成条件样本。实践中,分类器通常通过最小化经验损失函数训练得到。尽管现有统计理论在样本量足够大时保证良好泛化性能,但尚不清楚这种训练是否能产生有效的引导机制。本文在广泛使用的交叉熵损失背景下研究该问题。在分类器满足温和光滑性假设的前提下,我们证明:在每一步扩散模型中控制交叉熵,即可控制相应的引导误差。具体而言,实现条件KL散度ε²的分类器,其诱导的引导向量均方误差为~O(dε),忽略常数和对数因子。该结果给出了分类器引导扩散模型采样误差的上界,形式类似逆对数-索博列夫不等式。据我们所知,这是首个定量关联分类器训练与扩散模型引导对齐的理论结果,既解释了分类器引导的实证成功,也提供了选择有效引导分类器的合理指导。

原文摘要 · Abstract (English)

Classifier-guided diffusion models generate conditional samples by augmenting the reverse-time score with the gradient of the log-probability predicted by a probabilistic classifier. In practice, this classifier is usually obtained by minimizing an empirical loss function. While existing statistical theory guarantees good generalization performance when the sample size is sufficiently large, it remains unclear whether such training yields an effective guidance mechanism. We study this question in the context of cross-entropy loss, which is widely used for classifier training. Under mild smoothness assumptions on the classifier, we show that controlling the cross-entropy at each diffusion model step is sufficient to control the corresponding guidance error. In particular, probabilistic classifiers achieving conditional KL divergence $\varepsilon^2$ induce guidance vectors with mean squared error $\widetilde O(d \varepsilon )$, up to constant and logarithmic factors. Our result yields an upper bound on the sampling error of classifier-guided diffusion models and bears resemblance to a reverse log-Sobolev--type inequality. To the best of our knowledge, this is the first result that quantitatively links classifier training to guidance alignment in diffusion models, providing both a theoretical explanation for the empirical success of classifier guidance, and principled guidelines for selecting classifiers that induce effective guidance.

扩散模型分类器引导理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。