Langevin动态对得分函数误差不鲁棒，即使误差极小也难生成准确样本。
On the Robustness of Langevin Dynamics to Score Function Error
- 分析得分函数估计误差对Langevin动态的影响机制。
- 高维下任意多项式时间运行，分布与目标分布的总变差距离仍很大。
- 提醒谨慎使用带估计得分的Langevin动态，支持扩散模型优势。
我们研究了基于得分的生成模型对得分函数估计误差的鲁棒性。特别地，我们证明了在得分函数的$ L^2 $（更一般地$ L^p $）误差下，Langevin动态不具备鲁棒性。已知在$ L^2 $误差较小时，扩散模型可在多项式时间范围内，于较弱正则性假设下忠实采样目标分布。相比之下，本文结果表明：即使在高维简单分布下，对任意多项式时间范围运行的Langevin动态，其生成分布与目标分布之间的总变差（TV）距离仍会显著偏离，即便得分函数估计的$ L^2 $（或$ L^p $）误差可任意小。由于从数据中学习得分函数时此类误差不可避免，本研究进一步支持扩散模型优于Langevin动态，并警示在使用估计得分的Langevin动态时需谨慎。
原文摘要 · Abstract (English)
We consider the robustness of score-based generative modeling to errors in the estimate of the score function. In particular, we show that Langevin dynamics is not robust to the $L^2$ errors (more generally $L^p$ errors) in the estimate of the score function. It is well-established that with small $L^2$ errors in the estimate of the score function, diffusion models can sample faithfully from the target distribution under fairly mild regularity assumptions in a polynomial time horizon. In contrast, our work shows that even for simple distributions in high dimensions, Langevin dynamics run for any polynomial time horizon will produce a distribution far from the target distribution in Total Variation (TV) distance, even when the $L^2$ error (more generally $L^p$) of the estimate of the score function is arbitrarily small. Considering such an error in the estimate of the score function is unavoidable in practice when learning the score function from data, our results provide further justification for diffusion models over Langevin dynamics and serve to caution against the use of Langevin dynamics with estimated scores.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。