用自动微分实现高维隐变量高效积分,提升贝叶斯推断可扩展性。
Automatic Laplace Collapsed Sampling: Scalable Marginalisation of Latent Parameters via Automatic Differentiation
- 通过自动微分做最大后验与拉普拉斯近似,将高维隐变量压缩为标量
- 在多个模型上验证了证据计算的可行性,支持重尾分布改进
- 适合需要大规模贝叶斯推断且无梯度代码的用户
我们提出自动拉普拉斯坍缩采样(ALCS),一种基于自动微分的贝叶斯模型隐变量积分框架,结合嵌套采样以稳健高效地探索超参数空间。每次嵌套采样中的似然评估中,ALCS通过最大后验优化和拉普拉斯近似将高维隐变量 $z$ 坍缩为标量贡献,二者均使用自动微分实现。这使有效维度从 $d_θ+ d_z$ 降至仅 $d_θ$,无需手动推导梯度或海森矩阵,在高维设置下仍能进行贝叶斯证据计算,且模型定制成本极低。MAP优化与海森矩阵计算在GPU上并行于各存活点,具备实际可扩展性。我们还展示自动微分支持超越拉普拉斯的局部近似,如学生-t分布族,可提升重尾隐变量的证据估计精度。在涵盖层次模型、时间序列和离散似然模型的一系列基准测试中验证了ALCS,并确立了高斯近似适用范围。该方法还可实现事后有效样本量诊断,无需昂贵联合采样即可定位超参数空间中的失效区域。
原文摘要 · Abstract (English)
We present Automatic Laplace Collapsed Sampling (ALCS), a general framework for marginalising latent parameters in Bayesian models using automatic differentiation, which we combine with nested sampling to explore the hyperparameter space in a robust and efficient manner. At each nested sampling likelihood evaluation, ALCS collapses the high-dimensional latent variables $z$ to a scalar contribution via maximum a posteriori (MAP) optimisation and a Laplace approximation, both computed using autodiff. This reduces the effective dimension from $d_θ+ d_z$ to just $d_θ$, making Bayesian evidence computation tractable for high-dimensional settings without hand-derived gradients or Hessians, and with minimal model-specific engineering. The MAP optimisation and Hessian evaluation are parallelised across live points on GPU-hardware, making the method practical at scale. We also show that automatic differentiation enables local approximations beyond Laplace to parametric families such as the Student-$t$, which improves evidence estimates for heavy-tailed latents. We validate ALCS on a suite of benchmarks spanning hierarchical, time-series, and discrete-likelihood models and establish where the Gaussian approximation holds. This enables a post-hoc ESS diagnostic that localises failures across hyperparameter space without expensive joint sampling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。