用海森矩阵提升SMC²采样精度与效率
Hess-MC2: Sequential Monte Carlo Squared using Hessian Information and Second Order Proposals
- 引入目标函数的海森矩阵信息构造二阶提案分布
- 相比一阶方法，可容忍更大步长且降低重要权重方差
- 适合高维后验分布采样，尤其在计算资源充足时
在使用序列蒙特卡洛（SMC）进行贝叶斯推断时，后验近似精度与计算效率是两个关键问题。为应对计算需求，序列蒙特卡洛平方（SMC²）特别适用于高性能计算（HPC）环境。SMC²中提案分布的设计影响后验探索精度与效率，不良提案会导致重要性权重方差大及粒子退化。梅特罗波利斯-调整朗之万算法（MALA）利用梯度信息使粒子更倾向高概率区域。本文首次将二阶信息（即对数目标函数的海森矩阵）引入SMC²框架，提出基于海森矩阵的二阶提案。该方法不仅使用梯度（一阶导数），还利用目标分布的曲率（二阶导数）。在合成模型上的实验表明，相比其他提案，本方法在步长选择和后验近似精度方面均有显著优势。
原文摘要 · Abstract (English)
When performing Bayesian inference using Sequential Monte Carlo (SMC) methods, two considerations arise: the accuracy of the posterior approximation and computational efficiency. To address computational demands, Sequential Monte Carlo Squared (SMC$^2$) is well-suited for high-performance computing (HPC) environments. The design of the proposal distribution within SMC$^2$ can improve accuracy and exploration of the posterior as poor proposals may lead to high variance in importance weights and particle degeneracy. The Metropolis-Adjusted Langevin Algorithm (MALA) uses gradient information so that particles preferentially explore regions of higher probability. In this paper, we extend this idea by incorporating second-order information, specifically the Hessian of the log-target. While second-order proposals have been explored previously in particle Markov Chain Monte Carlo (p-MCMC) methods, we are the first to introduce them within the SMC$^2$ framework. Second-order proposals not only use the gradient (first-order derivative), but also the curvature (second-order derivative) of the target distribution. Experimental results on synthetic models highlight the benefits of our approach in terms of step-size selection and posterior approximation accuracy when compared to other proposals.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。