arXiv:2603.00772stat.MLcs.LG2026-03

提出统一框架,让生成模型更好处理重尾数据。

Generalizing Score-based generative models for Heavy-tailed Distributions

  • 用归一化流捕捉重尾特征,再作为扩散模型的初始先验。
  • 理论证明该方法在任意目标分布下均能收敛到真实分布。
  • 适合需要高精度建模长尾分布的研究者使用。

基于得分的生成模型(SGMs)在众多数据分布上表现优异,但将其推广至重尾分布仍是一个开放问题。尽管已有针对重尾分布的专用模型,其生成质量尚不明确且缺乏坚实的理论基础。本文通过两项理论贡献填补这一空白:首先,证明结合早停与合适初始化足以将扩散框架扩展至任意目标分布,建立了反向过程的适定性,并证明了近似扩散在KL散度下的收敛性;其次,为归一化流生成推导出新理论保证,在对流族施加弱条件且不假设目标分布尾部行为的情况下实现收敛。基于此,我们提出一个统一的重尾分布生成框架:先训练归一化流捕捉尾部特性,再将其作为SGM的初始化先验,由SGM进一步恢复细粒度结构细节。该设计在理论严谨的前提下融合两类模型的优势,克服了现有方法的局限。

原文摘要 · Abstract (English)

Score-based generative models (SGMs) have achieved remarkable empirical success, motivating their application to a broad range of data distributions. However, extending them to heavy-tailed targets remains a largely open problem. Although dedicated models for heavy-tailed distributions have been proposed, their generative fidelity remains unclear and they lack solid theoretical foundations, leaving important questions open in this regime. In this paper, we address this gap through two theoretical contributions. First, we show that combining early stopping with a suitable initialization is sufficient to extend the diffusion framework to any target distribution; in particular, we establish the well-posedness of the backward process and prove convergence of the approximated diffusion in KL divergence. Second, we derive novel theoretical guarantees for generation with normalizing flows, obtaining convergence results that hold under mild conditions on the flow family and without any assumption on the tail behavior of the target distribution. Building on these results, we propose a unified generative framework for heavy-tailed distributions: a normalizing flow is first trained to capture the tail behavior and is then used as an initialization prior for an SGM, which refines the samples by recovering fine-grained structural details. This design leverages the complementary strengths of the two model classes within a theoretically principled pipeline, overcoming the limitations of existing approaches.

生成模型重尾分布归一化流扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。