时间依赖的损失加权在流匹配与扩散模型中理论成立
Time dependent loss reweighting for flow matching and diffusion models is theoretically justified
- 损失与参数可同时依赖时间t和当前状态X_t
- 时间分布可选范围广，提升训练稳定性
- 适用于流匹配、扩散模型及编辑流，利于预测器构建
本文阐明，在生成器匹配（Generator Matching）框架下（涵盖连续、流形及离散空间中的大量流匹配与扩散模型），生成器的线性参数化及Bregman散度损失均可同时依赖当前状态X_t与时间t。我们证明损失对时间的期望可基于广泛的时间分布进行。该结论也适用于超出生成器匹配框架的编辑流（Edit Flows）。损失依赖时间，使实践中常用于稳定训练的时间依赖损失加权方案获得理论支持。此外，该依赖关系常简化X_1-预测器的设计，后者在某些场景下更优。文中通过实例展示了线性参数化与Bregman损失对t和X_t的依赖性。
原文摘要 · Abstract (English)
This brief note clarifies that, in Generator Matching (which subsumes a large family of flow matching and diffusion models over continuous, manifold, and discrete spaces), both the Bregman divergence loss and the linear parameterization of the generator can depend on both the current state $X_t$ and the time $t$, and we show that the expectation over time in the loss can be taken with respect to a broad class of time distributions. We also show this for Edit Flows, which falls outside of Generator Matching. That the loss can depend on $t$ clarifies that time-dependent loss weighting schemes, often used in practice to stabilize training, are theoretically justified when the specific flow or diffusion scheme is a special case of Generator Matching (or Edit Flows). It also often simplifies the construction of $X_1$-predictor schemes, which are sometimes preferred for model-related reasons. We show examples that rely upon the dependence of linear parameterizations, and of the Bregman divergence loss, on $t$ and $X_t$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。