解决流形数据生成中得分函数奇异问题，提升扩散模型采样精度
Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity
- 分解得分函数在切向与法向的结构，揭示多尺度奇异机制
- 提出Niso-DM与Tango-DM，分别用非各向同性噪声和切向损失优化
- 在复杂几何流形上显著改善生成质量，适合高维流形建模研究者
欧几里得扩散模型在多个领域生成建模中取得显著成功，近期研究已将其扩展至流形数据。本文不依赖特定流形结构，直接对一般流形结构数据进行欧氏扩散模型采样。我们揭示了嵌入空间中得分函数的多尺度奇异现象，阻碍了生成样本的准确性。通过沿流形切向与法向分解得分函数，进行了详尽的理论分析。为缓解奇异性和提升采样精度，提出两种新方法：(1) Niso-DM，利用非各向同性噪声降低得分函数尺度差异；(2) Tango-DM，仅使用切向损失函数训练得分函数的切向分量。数值实验表明，所提方法在具有复杂几何结构的多种流形分布上均取得优越性能。
原文摘要 · Abstract (English)
Euclidean diffusion models have achieved remarkable success in generative modeling across diverse domains, and they have been extended to manifold cases in recent advances. Instead of explicitly utilizing the structure of special manifolds as studied in previous works, in this paper we investigate direct sampling of the Euclidean diffusion models for general manifold-structured data. We reveal the multiscale singularity of the score function in the ambient space, which hinders the accuracy of diffusion-generated samples. We then present an elaborate theoretical analysis of the singularity structure of the score function by decomposing it along the tangential and normal directions of the manifold. To mitigate the singularity and improve the sampling accuracy, we propose two novel methods: (1) Niso-DM, which reduces the scale discrepancies in the score function by utilizing a non-isotropic noise, and (2) Tango-DM, which trains only the tangential component of the score function using a tangential-only loss function. Numerical experiments demonstrate that our methods achieve superior performance on distributions over various manifolds with complex geometries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。