提出更优的无海森采样算法,提升收敛速度与精度。
Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling
- 通过位置扩散增强动力学,不依赖凸性假设
- 理论证明收敛速度优于传统方法,且在有限精度下仍有效
- 适合机器学习中的贝叶斯推断等高维采样问题
无海森高分辨率(HFHR)动力学在欠阻尼朗之万动力学(ULD)基础上引入可逆位置扩散,用于解决机器学习中的采样问题。本文在位置庞加莱不等式、加权海森与拉普拉斯界及紧 Sobolev 嵌入条件下,建立了 HFHR 动力学的显式量化收缩率,无需势函数凸性假设。通过适配的时间-增广庞加莱不等式,获得优于 ULD 的收缩速率。还给出了弱解构造和自包含的谱证明以支持发散引理。针对基于 HFHR 动力学离散化的 HFHRMC 算法,利用路径空间吉尔萨诺夫论证,得到非渐近收敛界与显式迭代复杂度(总变差距离下),该界对任意 α≥0 与 γ>0 成立,并在 ULD 极限点保持正则性。优化迭代复杂度得正的、与精度相关的位置扩散参数,在高精度下其主导阶与最优 ULD 相同。相比已有工作,本方法的迭代复杂度更优。数值实验包括真实数据上的贝叶斯学习问题,验证了正 α 的效果及其优势。
原文摘要 · Abstract (English)
Hessian-free high-resolution (HFHR) dynamics augments underdamped Langevin dynamics (ULD) with reversible position diffusion for sampling problems that arise in machine learning. We establish an explicit quantitative contraction rate for HFHR dynamics under a position Poincaré inequality, weighted Hessian and Laplacian bounds, and a compact Sobolev embedding, where the potential function is not necessarily convex. An adapted time-augmented Poincaré inequality yields an explicit rate that improves upon the contraction rate of the underdamped Langevin dynamics. We also give a weak-solution construction and a self-contained spectral proof of the divergence lemma underlying the argument. For HFHR Monte Carlo (HFHRMC) algorithm, which is based on a discretization scheme of HFHR dynamics, we use a path-space Girsanov argument to obtain a non-asymptotic convergence bound and an explicit iteration complexity in total variation distance. The bounds hold for every $α\geq0$ and $γ>0$ and remain regular at the ULD endpoint. Optimizing the iteration complexity bound yields a positive, accuracy-dependent position-diffusion parameter at finite accuracy, while its leading high-accuracy order coincides with that of the optimized ULD endpoint. Our iteration complexity bound improves upon the existing work on HFHR algorithms. Numerical experiments including Bayesian learning problems on real data are provided to illustrate the effect of positive $α$ and its benefit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。