用学生分布改进扩散模型,更好生成极端罕见事件。
Heavy-Tailed Diffusion Models
- 改用学生t分布作为先验,设计适配重尾分布的去噪机制。
- 在气象数据上生成极端天气事件的能力显著优于传统模型。
- 仅需调整一个超参数,即可灵活控制尾部生成,兼容现有框架。
扩散模型在众多应用中达到顶尖生成质量,但其对重尾分布中稀有或极端事件的捕捉能力尚不明确。本文表明,采用标准高斯先验的传统扩散与流匹配模型无法有效建模重尾行为。为此,我们重新利用扩散框架进行重尾估计,引入多变量学生t分布。设计了专用扰动核,并基于条件学生t分布推导后向过程的去噪后验。受γ-散度启发,我们构建了适用于重尾去噪器的训练目标。所提框架通过单一标量超参数实现可控尾部生成,便于适配多种真实世界分布。作为具体实例,我们提出t-EDM和t-Flow,即采用学生t先验的扩散与流模型扩展。令人惊讶的是,该方法与标准高斯扩散模型高度兼容,仅需少量代码修改。实验证明,在高分辨率气象数据集上,我们的t-EDM和t-Flow在极端事件生成方面显著优于标准扩散模型。
原文摘要 · Abstract (English)
Diffusion models achieve state-of-the-art generation quality across many applications, but their ability to capture rare or extreme events in heavy-tailed distributions remains unclear. In this work, we show that traditional diffusion and flow-matching models with standard Gaussian priors fail to capture heavy-tailed behavior. We address this by repurposing the diffusion framework for heavy-tail estimation using multivariate Student-t distributions. We develop a tailored perturbation kernel and derive the denoising posterior based on the conditional Student-t distribution for the backward process. Inspired by $γ$-divergence for heavy-tailed distributions, we derive a training objective for heavy-tailed denoisers. The resulting framework introduces controllable tail generation using only a single scalar hyperparameter, making it easily tunable for diverse real-world distributions. As specific instantiations of our framework, we introduce t-EDM and t-Flow, extensions of existing diffusion and flow models that employ a Student-t prior. Remarkably, our approach is readily compatible with standard Gaussian diffusion models and requires only minimal code changes. Empirically, we show that our t-EDM and t-Flow outperform standard diffusion models in heavy-tail estimation on high-resolution weather datasets in which generating rare and extreme events is crucial.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。