扩散模型并非均匀欠拟合，而是在特定区域更精准逼近数据得分。
Selective Underfitting in Diffusion Models
- 提出选择性欠拟合概念：模型在部分输入空间更准确学习得分函数。
- 实验证明理想扩散模型在关键区域逼近得分，其余区域则故意欠拟合。
- 为理解生成性能与泛化能力提供新视角，适合研究生成模型机制者阅读。
扩散模型已成为跨领域生成建模的主流范式。训练中它们学习得分函数，用于推理阶段生成样本。这引发一个基本但未解决的问题：模型究竟学到了什么样的得分？理论上，若完全匹配数据空间中的经验得分，模型将仅复现训练数据，无法生成新样本。近期研究认为，扩散模型因训练时归纳偏置而对经验得分存在整体欠拟合。本文进一步细化该观点，提出选择性欠拟合：更好的扩散模型并非在所有区域欠拟合，而是在输入空间的某些区域更精确地逼近得分，其他区域则刻意欠拟合。我们刻画了这些区域，并设计了实证干预加以验证。结果表明，选择性欠拟合是理解扩散模型的关键，为模型泛化与生成性能提供了可检验的新见解。
原文摘要 · Abstract (English)
Diffusion models have emerged as the principal paradigm for generative modeling across various domains. During training, they learn the score function, which in turn is used to generate samples at inference. They raise a basic yet unsolved question: which score do they actually learn? In principle, a diffusion model that matches the empirical score in the entire data space would simply reproduce the training data, failing to generate novel samples. Recent work addresses this question by arguing that diffusion models underfit the empirical score due to training-time inductive biases. In this work, we refine this perspective, introducing the notion of selective underfitting: instead of underfitting the score everywhere, better diffusion models more accurately approximate the score in certain regions of input space, while underfitting it in others. We characterize these regions and design empirical interventions to validate our perspective. Our results establish that selective underfitting is essential for understanding diffusion models, yielding new, testable insights into their generalization and generative performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。