研究重尾分布下扩散模型的评分估计与采样精度,突破了传统轻尾假设限制。
Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees
- 基于核密度估计,分析重尾目标分布的评分估计最优率
- 在指数尾部下采样误差达最优率(含对数因子),多项式尾部依赖尾参数γ
- 为实际数据中常见重尾场景提供了理论支撑,适合理论研究者参考
基于评分的扩散模型已成为生成建模的强大框架,其中评分估计是核心统计瓶颈。现有评分估计的理论保证大多针对轻尾分布或依赖紧支集等限制性假设,而这些假设在实际重尾数据中常不成立。本文研究目标分布为重尾且属于光滑参数β>0的Sobolev类时,经典的(高斯)评分扩散模型。考虑指数尾和多项式尾,分别由尾参数γ刻画。通过核密度估计,推导出评分估计的尖锐极小极大率,揭示定性差异:指数尾下速率接近轻尾情形(仅含多对数因子),而多项式尾下速率显式依赖γ。进一步给出关联连续反向动态的采样保证。在总变差距离下,指数尾下生成分布收敛速率达最小极大最优率n^{-β/(2β+d)}(含对数因子),多项式尾下则为γ相关速率。后者是否为最小极大最优仍为开放问题。这些结果刻画了重尾目标下评分估计与采样精度的统计极限,将扩散理论拓展至轻尾以外的场景。
原文摘要 · Abstract (English)
Score-based diffusion models have become a powerful framework for generative modeling, with score estimation as a central statistical bottleneck. Existing guarantees for score estimation largely focus on light-tailed targets or rely on restrictive assumptions such as compact support, which are often violated by heavy-tailed data in practice. In this work, we study conventional (Gaussian) score-based diffusion models when the target distribution is heavy-tailed and belongs to a Sobolev class with smoothness parameter $β>0$. We consider both exponential and polynomial tail decay, indexed by a tail parameter $γ$. Using kernel density estimation, we derive sharp minimax rates for score estimation, revealing a qualitative dichotomy: under exponential tails, the rate matches the light-tailed case up to polylogarithmic factors, whereas under polynomial tails the rate depends explicitly on $γ$. We further provide sampling guarantees for the associated continuous reverse dynamics. In total variation, the generated distribution converges at the minimax optimal rate $n^{-β/(2β+d)}$ under exponential tails (up to logarithmic factors), and at a $γ$-dependent rate under polynomial tails. Whether the latter sampling rate is minimax optimal remains an open question. These results characterize the statistical limits of score estimation and the resulting sampling accuracy for heavy-tailed targets, extending diffusion theory beyond the light-tailed setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。