提出更优的点云扩散模型旋转对齐方法,提升生成质量。
Matching the Optimal Denoiser in Point Cloud Diffusion with (Improved) Rotational Alignment
- 基于矩阵Fisher分布建模最优去噪器的旋转特性。
- 证明现有对齐方法在低噪声下为零阶近似,效果已足够好。
- 新方法在小噪声下逼近最优去噪器,适合分子蛋白生成任务。
扩散模型通过逆转加噪过程来学习数据分布。训练时需学习在不同噪声水平下的去噪能力。针对点云数据(如分子、蛋白质)无固定朝向的问题,通常采用SO(3)上均匀采样的随机旋转进行数据增强。去噪预测结果常通过Kabsch-Umeyama算法对齐真值样本后再计算损失。然而该对齐步骤的影响尚未被充分研究。本文表明,最优去噪器可由SO(3)上的矩阵Fisher分布表达;对齐即采样该分布的众数,在低噪声下为零阶近似,解释了其有效性。在此基础上,我们推导出小噪声极限下更优的去噪器近似方法。实验显示,对齐在扩散模型训练最关键的噪声水平下已是‘足够好’的近似。
原文摘要 · Abstract (English)
Diffusion models are a popular class of generative models trained to reverse a noising process starting from a target data distribution. Training a diffusion model consists of learning how to denoise noisy samples at different noise levels. When training diffusion models for point clouds such as molecules and proteins, there is often no canonical orientation that can be assigned. To capture this symmetry, the true data samples are often augmented by transforming them with random rotations sampled uniformly over $SO(3)$. Then, the denoised predictions are often rotationally aligned via the Kabsch-Umeyama algorithm to the ground truth samples before computing the loss. However, the effect of this alignment step has not been well studied. Here, we show that the optimal denoiser can be expressed in terms of a matrix Fisher distribution over $SO(3)$. Alignment corresponds to sampling the mode of this distribution, and turns out to be the zeroth order approximation for small noise levels, explaining its effectiveness. We build on this perspective to derive better approximators to the optimal denoiser in the limit of small noise. Our experiments highlight that alignment is often a `good enough' approximation for the noise levels that matter most for training diffusion models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。