扩散模型比分类模型学得更全面，因去噪目标鼓励平衡表征。
On the Feature Learning in Diffusion Models
- 通过去噪目标设计特征学习框架，对比生成与分类模型。
- 实验表明扩散模型学习更均衡的特征，分类模型偏重易学模式。
- 适合研究生成模型机制或对表征学习感兴趣的读者。
扩散模型在生成建模中的成功引发了对其理论基础的广泛关注。本文提出一种特征学习框架，用于分析和比较扩散模型与传统分类模型的训练动态。理论分析表明，由于去噪目标，扩散模型倾向于学习更均衡、更全面的数据表征；而具有相似架构的分类模型则更关注数据中的特定模式，常聚焦于易于学习的成分。为验证这些理论洞见，我们在合成数据集和真实世界数据集上进行了多项实验，实证结果支持了上述发现，并凸显了扩散模型在特征学习动态上的独特性。
原文摘要 · Abstract (English)
The predominant success of diffusion models in generative modeling has spurred significant interest in understanding their theoretical foundations. In this work, we propose a feature learning framework aimed at analyzing and comparing the training dynamics of diffusion models with those of traditional classification models. Our theoretical analysis demonstrates that diffusion models, due to the denoising objective, are encouraged to learn more balanced and comprehensive representations of the data. In contrast, neural networks with a similar architecture trained for classification tend to prioritize learning specific patterns in the data, often focusing on easy-to-learn components. To support these theoretical insights, we conduct several experiments on both synthetic and real-world datasets, which empirically validate our findings and highlight the distinct feature learning dynamics in diffusion models compared to classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。