图像扩散模型的局部性源于数据本身的统计特性,而非网络结构设计。
Locality in Image Diffusion Models Emerges from Data Statistics
- 通过线性去噪器验证:局部性是数据相关性的自然结果。
- 真实图像数据中像素相关性直接导致去噪时仅依赖局部区域。
- 适用于理解生成机制、优化去噪器的设计者。
近期研究表明,图像扩散模型的泛化能力源于训练后神经网络的局部性特征。具体而言,在去噪特定像素时,模型仅依赖该像素附近的有限邻域,这一特性与模型生成新图像的能力密切相关。由于局部性对泛化至关重要,理解其成因及其调控因素极为关键。本文提出证据表明,深度扩散模型中的局部性是图像数据集的统计属性,并非卷积神经网络的归纳偏置所致。我们证明,最优参数化线性去噪器也表现出与深度神经去噪器相似的局部性。理论与实验均表明,这种局部性直接源于图像数据集中存在的像素相关性。此外,专用数据集上的局部性模式显著不同,近似于数据协方差矩阵的主成分。基于这些洞察,我们构建了一个解析式去噪器,其预测得分比以往人工设计的方法更接近深度扩散模型的得分。核心结论是:尽管神经网络架构影响生成质量,但其主要作用是捕捉数据内在的局部性模式。
原文摘要 · Abstract (English)
Recent work has shown that the generalization ability of image diffusion models arises from the locality properties of the trained neural network. In particular, when denoising a particular pixel, the model relies on a limited neighborhood of the input image around that pixel, which, according to the previous work, is tightly related to the ability of these models to produce novel images. Since locality is central to generalization, it is crucial to understand why diffusion models learn local behavior in the first place, as well as the factors that govern the properties of locality patterns. In this work, we present evidence that the locality in deep diffusion models emerges as a statistical property of the image dataset and is not due to the inductive bias of convolutional neural networks, as suggested in previous work. Specifically, we demonstrate that an optimal parametric linear denoiser exhibits similar locality properties to deep neural denoisers. We show, both theoretically and experimentally, that this locality arises directly from pixel correlations present in the image datasets. Moreover, locality patterns are drastically different on specialized datasets, approximating principal components of the data's covariance. We use these insights to craft an analytical denoiser that better matches scores predicted by a deep diffusion model than prior expert-crafted alternatives. Our key takeaway is that while neural network architectures influence generation quality, their primary role is to capture locality patterns inherent in the data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。