arXiv:2505.04417cs.LGcs.NA2025-05被引 6

通过局部结构设计,让扩散模型在低维空间中训练,降低数据需求。

Localized Diffusion Models

  • 用局部化神经网络估计得分函数,减少样本需求。
  • 中等局部半径下,性能优于传统扩散模型。
  • 适合大规模生成任务,支持并行训练,效率更高。

扩散模型是各类生成任务的前沿工具,但其训练需估计高维得分函数,本质上受维度灾难制约。本文关注目标分布中的局部结构——即变量间稀疏的条件依赖关系。在局部结构下,得分函数实际为低维,可由局部化神经网络以更少样本有效估计。为此提出局部扩散模型,采用局部化得分匹配损失,在局部假设空间中训练得分函数。理论与数值结果表明:适当局部半径可平衡统计误差与局部化误差,显著提升整体性能;同时局部结构支持并行训练,有利于大规模应用。

原文摘要 · Abstract (English)

Diffusion models are state-of-the-art tools for various generative tasks. Yet training these models involves estimating high-dimensional score functions, which in principle suffers from the curse of dimensionality. It is therefore important to understand how low-dimensional structure in the target distribution can be exploited in these models. Here we consider locality structure, which describes certain sparse conditional dependencies among the target random variables. Given some locality structure, the score function is effectively low-dimensional, so that it can be estimated by a localized neural network with significantly reduced sample complexity. This observation motivates the localized diffusion model, where a localized score matching loss is used to train the score function within a localized hypothesis space. We prove that such localization enables diffusion models to circumvent the curse of dimensionality, at the price of additional localization error. Under realistic sample size scaling, we then show both theoretically and numerically that a moderate localization radius can balance the statistical and localization errors, yielding better overall performance. Localized structure also facilitates parallel training, making localized diffusion models potentially more efficient for large-scale applications.

扩散模型局部结构低维建模高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。