通过分解得分函数，让模型更高效地学习流形上的数据分布。
Manifold Aware Denoising Score Matching (MAD)
- 将得分函数拆分为已知成分与待学成分，利用已知部分隐含流形信息。
- 在旋转矩阵和离散分布上验证，计算开销低且精度高。
- 适合研究流形学习或需高效建模的机器学习开发者。
设计定义在流形上的分布学习方法时，关键挑战在于避免显式学习流形结构，使学习重点聚焦于流形内的数据分布。然而，现有方法常导致计算成本过高。本文提出对环境空间中的去噪得分匹配进行简单修改，通过隐式包含流形信息来减轻学习负担，同时保持计算效率。具体而言，我们将得分函数分解为一个已知成分 $s^{base}$ 与一个余项成分 $s-s^{base}$（即学习目标），其中前者隐含了数据流形的位置信息。我们为旋转矩阵分布和离散分布等重要情形推导出 $s^{base}$ 的解析形式，并在这些场景中验证了该方法的有效性。
原文摘要 · Abstract (English)
A major focus in designing methods for learning distributions defined on manifolds is to alleviate the need to implicitly learn the manifold so that learning can concentrate on the data distribution within the manifold. However, accomplishing this often leads to compute-intensive solutions. In this work, we propose a simple modification to denoising score-matching in the ambient space to implicitly account for the manifold, thereby reducing the burden of learning the manifold while maintaining computational efficiency. Specifically, we propose a simple decomposition of the score function into a known component $s^{base}$ and a remainder component $s-s^{base}$ (the learning target), with the former implicitly including information on where the data manifold resides. We derive known components $s^{base}$ in analytical form for several important cases, including distributions over rotation matrices and discrete distributions, and use them to demonstrate the utility of this approach in those cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。