扩散模型通过对数域平滑实现几何自适应，提升数据流形上的泛化能力。
Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
- 在对数密度域进行平滑，使得分函数沿数据流形方向平滑。
- 平滑操作能有效控制扩散模型的泛化流形，提升生成质量。
- 为扩散模型的几何自适应性提供了理论与实证支持，适合生成模型研究者。
扩散模型已达到顶尖性能，在多个领域展现出卓越的泛化能力。然而，其背后机制仍不完全清楚。一种主流假说基于流形假设，认为其成功源于对数据低维几何结构的适应。本文通过分析得分匹配中的学习问题形式，验证了该假说。理论与实证结果表明，对经验得分匹配目标的最小化进行平滑——等价于在对数密度域中平滑——可产生沿数据流形方向的平滑效果。此外，我们证明可通过选择合适的平滑方式，调控扩散模型所依赖的泛化流形。
原文摘要 · Abstract (English)
Diffusion models have achieved state-of-the-art performance, demonstrating remarkable generalisation capabilities across diverse domains. However, the mechanisms underpinning these strong capabilities remain only partially understood. A leading conjecture, based on the manifold hypothesis, attributes this success to their ability to adapt to low-dimensional geometric structure within the data. This work provides evidence for this conjecture, focusing on how such phenomena could result from the formulation of the learning problem through score matching. We inspect the role of implicit regularisation by investigating the effect of smoothing minimisers of the empirical score matching objective. Our theoretical and empirical results confirm that smoothing the score function -- or equivalently, smoothing in the log-density domain -- produces smoothing tangential to the data manifold. In addition, we show that the manifold along which the diffusion model generalises can be controlled by choosing an appropriate smoothing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。