研究扩散模型在低维流形数据上的学习曲线，发现线性结构可显著降低样本需求。
Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Data
- 用随机特征网络建模得分函数，分析高维极限下的训练与测试误差
- 线性流形下样本复杂度与内在维度线性相关，而非环境维度
- 非线性流形时结构优势减弱，说明数据结构类型影响关键
我们研究了当数据分布支持在低维流形上且得分函数由随机特征神经网络参数化时，去噪得分匹配——扩散模型的训练任务——的理论行为。在高维极限下，我们推导出测试、训练和得分误差的渐近精确表达式。分析表明，对于线性流形，学习得分函数所需的样本复杂度与流形的内在维度呈线性关系，而非环境维度。令人惊讶的是，一旦数据为非线性流形，低维结构的优势便开始减弱。这些结果表明，扩散模型可从结构化数据中获益；然而，这种依赖关系取决于结构的具体类型，且极为微妙复杂。
原文摘要 · Abstract (English)
We study the theoretical behavior of denoising score matching--the learning task associated to diffusion models--when the data distribution is supported on a low-dimensional manifold and the score is parameterized using a random feature neural network. We derive asymptotically exact expressions for the test, train, and score errors in the high-dimensional limit. Our analysis reveals that, for linear manifolds the sample complexity required to learn the score function scales linearly with the intrinsic dimension of the manifold, rather than with the ambient dimension. Perhaps surprisingly, the benefits of low-dimensional structure starts to diminish once we have a non-linear manifold. These results indicate that diffusion models can benefit from structured data; however, the dependence on the specific type of structure is subtle and intricate.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。