扩散模型能自适应高维多簇数据的内在结构,理论证明其生成误差与簇内维度相关。
Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification

- 将去噪过程视为动态贝叶斯分类器,后验概率在信号噪声比达Θ(log(KD)/D)时集中于单个簇
- KL误差界线性依赖于最大簇内维度,即使簇数K随维数D多项式增长也成立
- 适用于具有异质低秩协方差的多模态数据,突破传统全维分析局限
扩散模型在生成建模中的成功促使了理论研究,包括量化误差界和对去噪不同阶段的定性分析。本文将两者结合,研究扩散模型对高维多簇数据($bR^D$中多个簇,每簇具独立低维结构,簇间分离度依赖$D$)几何结构的自适应能力。采用$K$-混合高斯分布作为典型框架,建立两个理论结果:第一,将去噪解释为动态贝叶斯分类器,混合得分是簇级得分的后验加权平均;当信噪比达到Θ(log(KD)/D)时,后验类别概率以高概率集中在单一簇上。第二,通过分别分析去噪的混合与簇确认阶段,证明KL误差界与最大簇内维度呈线性关系(对数因子内),即使$K$随$D$多项式增长也成立。该结果优于基于环境维度的界,并将已有低维自适应分析拓展至具有异质近似低秩协方差的多模态分布。
原文摘要 · Abstract (English)
The empirical success of diffusion models in generative modelling has motivated theoretical work, including quantitative error bounds and qualitative analyses that characterise the different phases of denoising. We bring these two areas together by studying the adaptivity of diffusion models to the structured geometry of multimodal high-dimensional data that consists of multiple clusters in $\mathbb{R}^D$, each with its own low-dimensional structure, and inter-cluster separation depending on $D$. We employ $K$-mixture Gaussian distributions as a canonical framework to capture this geometry and establish two theoretical results. First, we interpret denoising as a dynamical Bayesian classifier: the mixture score is a posterior-weighted average of cluster-wise scores, and we show that, with high probability, the posterior class probabilities concentrate on a single cluster once the signal-to-noise ratio reaches the scale $Θ(\log (KD)/D)$. Second, by separately analysing the denoising process in its mixing and cluster-commitment phases, we prove that the KL error bound depends linearly on the maximum intrinsic dimension of a cluster, up to a logarithmic factor, even when $K$ grows polynomially with $D$. This improves on ambient-dimensional bounds and extends existing low-dimensional adaptivity analyses to multimodal distributions with heterogeneous, approximately low-rank covariances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。