提出低阶采样新方法,解决无梯度高维非对数凸分布采样难题
Zeroth-Order Non-Log-Concave Sampling with Variance Reduction and Applications to Inverse Problems

- 用方差缩减的零阶梯度估计提升采样稳定性
- 首次给出非渐近收敛保证,误差控制在ε相对费希尔信息内
- 适用于黑箱逆问题,尤其适合已有生成先验的场景
在梯度不可得或计算成本过高的黑箱设置下,从高维非对数凸分布中采样仍是机器学习中的基本挑战。尽管朗之万动力学在可获取梯度时提供合理框架,其扩展至黑箱场景时面临高方差且缺乏非渐近收敛保证的问题。为此,本文提出一种方差缩减的零阶朗之万采样方法,采用新型梯度估计器显著降低经典批量零阶估计器的方差,并消除对批量大小的不利维度依赖,实现高效稳定采样。我们首次建立了基于ε-相对费希尔信息的非渐近收敛性理论,并在庞卡雷不等式假设下获得平方总变差距离的收敛保证。进一步提出ZO-APMC算法,用于具有预训练得分生成先验的黑箱逆问题后验采样,首次为该类方法提供非渐近收敛性证明。通过合成实验验证理论,并在实际线性和非线性逆问题上展示优异性能。
原文摘要 · Abstract (English)
Sampling from high-dimensional, non-log-concave distributions with unnormalized densities remains a fundamental challenge in machine learning, particularly in black-box settings where gradient information is inaccessible or computationally prohibitive. While Langevin dynamics provides a principled framework for sampling when gradients are accessible, its extension to the black-box settings suffers from high variance and lacks non-asymptotic convergence guarantees for non-log-concave sampling. To address these limitations, we propose a variance-reduced zeroth-order Langevin sampling method. Our method employs a gradient estimator that substantially reduces the variance of the classical batched zeroth-order estimator and eliminates the unfavorable dimensional dependence of the batch size required for accurate estimation, enabling practical and stable sampling. We establish the first non-asymptotic convergence guarantees for zeroth-order non-log-concave sampling in terms of $\varepsilon$-relative Fisher information, and, under a Poincaré inequality assumption, squared total variation distance. We further propose ZO-APMC, a posterior sampling algorithm for black-box inverse problems with pre-trained score-based generative priors, establishing the first non-asymptotic convergence guarantees for such methods. We validate our theory through synthetic experiments and demonstrate strong empirical performance on practical linear and nonlinear inverse problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。