挑战扩散模型高维能力的理论根基，揭示其真实工作机理。
Rethinking Diffusion Model in High Dimension
- 发现高维稀疏下目标函数退化为单样本学习
- 推导出无需统计概念的统一推断框架
- 适合对扩散模型原理感兴趣的科研人员
维度诅咒是统计概率模型不可避免的挑战，但扩散模型似乎克服了这一限制，在高维数据生成中取得了显著成果。扩散模型假设能够学习底层概率分布的统计量，从而通过采样生成真实样本。但这是否真的成立？我们基于以下观察提出质疑：1）在高维稀疏场景下，扩散模型的目标函数从多个样本的加权和退化为单一样本，这可能阻碍模型有效学习后验、得分或速度场等关键统计量；2）大多数推理方法可统一于一个不依赖统计概念的简单框架，该框架与退化的目标函数一致，为推理过程提供了新颖且直观的视角。
原文摘要 · Abstract (English)
Curse of Dimensionality is an unavoidable challenge in statistical probability models, yet diffusion models seem to overcome this limitation, achieving impressive results in high-dimensional data generation. Diffusion models assume that they can learn the statistical quantities of the underlying probability distribution, enabling sampling from this distribution to generate realistic samples. But is this really how they work? We argue not, based on the following observations: 1) In high-dimensional sparse scenarios, the fitting target of the diffusion model's objective function degrades from a weighted sum of multiple samples to a single sample, which we believe hinders the model's ability to effectively learn essential statistical quantities such as posterior, score, or velocity field. 2) Most inference methods can be unified within a simple framework which involves no statistical concepts, aligns with the degraded objective function, and provides an novel and intuitive perspective on the inference process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。