arXiv:2507.01516cs.LG2025-07中稿 · ECML 2025被引 3

对比扩散模型的损失函数,揭示其原理差异与适用场景。

Loss Functions in Diffusion Models: A Comparative Study

  • 从变分下界统一框架分析不同损失函数的理论关系
  • 实证发现损失选择影响生成质量与似然估计性能
  • 帮助研究者按目标选损失函数,提升模型设计效率

扩散模型已成为强大的生成模型,激发了对其内在机制的广泛研究。其中关键问题之一是模型应采用何种损失函数进行训练。近年来文献中提出了多种公式,其来源各异,既有联系也存在关键差异。本文深入探讨了不同目标函数与对应损失函数的内在逻辑,系统性地梳理它们之间的关系,并在变分下界框架下实现统一。结合实证研究,揭示了这些目标函数在性能上出现分歧的条件及其背后驱动因素。此外,评估了目标选择对模型生成高质量样本和准确估计似然能力的影响。本研究为扩散模型中的损失函数提供了统一理解,有助于未来更高效、目标导向的模型设计。

原文摘要 · Abstract (English)

Diffusion models have emerged as powerful generative models, inspiring extensive research into their underlying mechanisms. One of the key questions in this area is the loss functions these models shall train with. Multiple formulations have been introduced in the literature over the past several years with some links and some critical differences stemming from various initial considerations. In this paper, we explore the different target objectives and corresponding loss functions in detail. We present a systematic overview of their relationships, unifying them under the framework of the variational lower bound objective. We complement this theoretical analysis with an empirical study providing insights into the conditions under which these objectives diverge in performance and the underlying factors contributing to such deviations. Additionally, we evaluate how the choice of objective impacts the model ability to achieve specific goals, such as generating high-quality samples or accurately estimating likelihoods. This study offers a unified understanding of loss functions in diffusion models, contributing to more efficient and goal-oriented model designs in future research.

扩散模型损失函数生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。