提出首个非对数凸采样统一分析,用方差减少提升高维逆问题采样质量。
Variance Reduction for Non-Log-Concave Sampling with Applications to Inverse Problems

- 基于方差减少的梯度估计统一处理非对数凸分布采样
- 在固定计算预算下,相对Fisher信息与总变差距离收敛更快
- 适用于基于得分生成先验的图像逆问题,实测样本质量显著提升
在机器学习中,从高维、非对数凸分布中采样且密度未归一化是一项基本挑战,尤其当势能函数精确梯度不可得,需通过具有高方差的随机梯度近似时。尽管如带动量的SGD、STORM和PAGE等方差减少技术在非凸优化中表现优异,但其在非对数凸分布采样中的作用尚未被充分研究。本文首次建立这些估计器在非对数凸分布采样中的统一分析框架,证明了在ε-相对Fisher信息意义下更优的非渐近收敛速率,并在Poincaré不等式假设下,实现了平方总变差距离的改进收敛。进一步证明了弱收敛到目标分布。还将方法扩展至使用得分生成先验求解逆问题。实验验证理论,在固定每轮梯度计算量条件下,方差减少技术持续提升了两类标准成像应用中的样本质量。
原文摘要 · Abstract (English)
Sampling from high-dimensional, non-log-concave distributions with unnormalized densities is a fundamental challenge in machine learning, particularly when the exact gradient of the potential is unavailable and must be approximated via stochastic gradients that exhibit high variance under a fixed budget of gradient computations per iteration. Although variance reduction techniques such as SGD with momentum, STORM, and PAGE have demonstrated improved convergence properties in non-convex optimization, their implications for sampling from non-log-concave distributions remain largely unexplored. In this work, we develop the first unified analysis of these estimators for sampling from non-log-concave distributions. We establish improved non-asymptotic convergence rates in $\varepsilon$-relative Fisher information and, under a Poincaré inequality assumption, in squared total variation distance, and further prove weak convergence to the target distribution. We extend our analysis to solving inverse problems with score-based generative priors. We empirically validate our theory and demonstrate that, under a fixed gradient computations per iteration, variance-reduction techniques consistently improve sample quality in two standard imaging applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。