用学生t分布改进扩散模型，让噪声自适应调节更拟合长尾数据。
Self-Regulating Annealing in Heavy-Tailed Diffusion Models
- 用状态依赖的扩散系数实现噪声自调节采样。
- 在长尾数据上生成样本的尾部更准确，提升分布拟合度。
- 适合研究长尾生成、扩散模型改进的学者和工程师。
扩散模型已成为深度生成建模的主流框架。尽管标准高斯形式在理论上方便，但其对长尾数据的适用性尚不明确。为此，长尾扩散模型（HTDM）通过将高斯分布替换为学生t分布，提升了对长尾数据的尾部拟合能力。虽然基于随机微分方程（SDE）的采样在HTDM中是可行的，但尚未充分探索。本文提出一种基于SDE的HTDM采样器，显式引入状态依赖的扩散系数。该状态依赖自然诱导出自调节退火机制，通过自适应调节有效噪声尺度实现。我们从理论上分析该机制，并实验验证其对再现长尾分布样本的必要性。
原文摘要 · Abstract (English)
Diffusion models have emerged as a leading framework for deep generative modeling. While the standard Gaussian formulation is theoretically convenient, its suitability for heavy-tailed datasets remains unclear. To address this, heavy-tailed diffusion models (HTDMs) extend the standard formulation by replacing the Gaussian distribution with a Student's t-distribution, thereby improving tail fidelity on heavy-tailed datasets. Although stochastic differential equation (SDE)-based sampling is possible in HTDMs, it has not been fully explored. In this paper, we propose an SDE-based sampler for HTDMs that explicitly incorporates a state-dependent diffusion coefficient. This state dependence naturally induces a self-regulating annealing mechanism by adaptively modulating the effective noise scale. We theoretically explore this mechanism and experimentally verify its necessity for reproducing samples from a heavy-tailed distribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。