arXiv:2506.10982cs.LGcs.AI2025-06NeurIPS被引 7

重新审视扩散桥采样器的损失函数,发现rKL-LD更优且更稳定。

Rethinking Losses for Diffusion Bridge Samplers

  • 用log-derivative trick替代重参数化,改进rKL损失在扩散桥中的优化目标。
  • 实验表明rKL-LD在多个挑战性基准上均优于LV损失,性能更佳。
  • rKL-LD需调参少、训练更稳定,适合实际应用与快速部署。

扩散桥是一类有前景的深度学习采样方法,用于从非归一化分布中采样。近期研究显示,在使用重参数化技巧计算rKL梯度时,对数方差(LV)损失始终优于反向Kullback-Leibler(rKL)损失。然而,当扩散过程不可学习时,基于策略的LV损失与rKL损失在梯度上等价;这一等价性在扩散桥或学习扩散系数的情况下不成立。基于此,我们指出,对于扩散桥,LV损失无法像rKL损失那样通过数据处理不等式获得合理的优化动机。分析表明,结合log-derivative trick的rKL损失(rKL-LD)不仅避免了这些概念问题,还持续优于LV损失。在多种类型的扩散桥和挑战性基准上的实验结果表明,采用rKL-LD训练的采样器表现更优。从实践角度看,rKL-LD所需超参数调优显著减少,训练行为更稳定。

原文摘要 · Abstract (English)

Diffusion bridges are a promising class of deep-learning methods for sampling from unnormalized distributions. Recent works show that the Log Variance (LV) loss consistently outperforms the reverse Kullback-Leibler (rKL) loss when using the reparametrization trick to compute rKL-gradients. While the on-policy LV loss yields identical gradients to the rKL loss when combined with the log-derivative trick for diffusion samplers with non-learnable forward processes, this equivalence does not hold for diffusion bridges or when diffusion coefficients are learned. Based on this insight we argue that for diffusion bridges the LV loss does not represent an optimization objective that can be motivated like the rKL loss via the data processing inequality. Our analysis shows that employing the rKL loss with the log-derivative trick (rKL-LD) does not only avoid these conceptual problems but also consistently outperforms the LV loss. Experimental results with different types of diffusion bridges on challenging benchmarks show that samplers trained with the rKL-LD loss achieve better performance. From a practical perspective we find that rKL-LD requires significantly less hyperparameter optimization and yields more stable training behavior.

扩散模型采样器损失函数优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。