提出可信赖的不确定性量化方法,提升隐变量模型推断精度
Large-scale Uncertainty Quantification for Latent Variable Models Using Subsampling Markov Chain Monte Carlo
- 基于子采样马尔可夫链蒙特卡洛,构建联合渐近理论
- 揭示全局参数与隐变量的跳变-扩散耦合动态特性
- 给出超参数调优准则,适用于需要可靠不确定性的场景
随机梯度朗之万动力学结合吉布斯更新(SGLD--Gibbs)为隐变量模型中的贝叶斯推断提供了高度可扩展的方法。然而,如何以合理方式调节算法超参数以确保不确定性估计具有统计意义仍不明确。本文通过建立SGLD--Gibbs的统计尺度极限理论,推导出在适当时空缩放下全局参数与隐变量的联合渐近极限:全局参数收敛至扩散型极限,而每个隐变量收敛至跳跃过程,反映了间歇性吉布斯更新的作用。该联合跳变-扩散结构揭示了隐变量随机性对全局参数稳态分布的影响。我们利用此结果提出显式超参数调优指导,确保不确定性量化有效。数值实验表明,采用该调优方案的SGLD--Gibbs在参数估计、不确定性量化和预测性能上均优于随机变分推断。
原文摘要 · Abstract (English)
Stochastic gradient Langevin dynamics combined with Gibbs updates (SGLD--Gibbs) provides a highly scalable approach to approximate Bayesian inference in latent variable models. However, it remains unclear how to tune the algorithm's hyperparameters in a principled manner to ensure the uncertainty estimates are statistically meaningful. In this work, we address this gap in tuning guidance by developing a statistical scaling limit theory for SGLD--Gibbs. We derive a joint asymptotic limit for the global parameters and latent variables under appropriate space-time rescaling. We show that global parameters converge to a diffusion-type limit, while each latent variable converges to a jump process, reflecting the use of intermittent Gibbs updates. This joint jump-diffusion structure reveals how latent-variable randomness contributes to the stationary distribution of the global parameters. We leverage our results to propose explicit guidance on hyperparameter tuning for SGLD--Gibbs that ensures meaningful uncertainty quantification. Numerical experiments show that SGLD--Gibbs with our tuning guidance leads to better parameter estimates, uncertainty quantification, and predictive performance than stochastic variational inference.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。