提出新采样方法，提升贝叶斯推断在小批量下的稳定性。
Robust Stochastic Gradient Posterior Sampling with Lattice Based Discretisation
- 通过格点随机游走设计更新协方差，仅在非对角线引入噪声
- 在小批量和重尾梯度噪声下仍保持稳定，预测性能更优
- 适合对鲁棒性要求高的贝叶斯学习场景
随机梯度马尔可夫链蒙特卡洛方法能实现可扩展的贝叶斯后验采样，但通常对小批量大小和梯度噪声敏感。为此，我们提出随机梯度格点随机游走（SGLRW），作为格点随机游走离散化的扩展。与传统的随机梯度朗之万动态（SGLD）不同，SGLRW仅通过更新协方差的非对角元素引入随机噪声，从而在保持渐近正确性的同时，显著增强对小批量大小的鲁棒性。此外，我们还分析了使用梯度裁剪的SGLD自然变体。在贝叶斯回归和分类任务上的实验表明，当SGLD失效时，SGLRW依然稳定，且在预测性能上达到或超过现有方法。
原文摘要 · Abstract (English)
Stochastic-gradient MCMC methods enable scalable Bayesian posterior sampling but often suffer from sensitivity to minibatch size and gradient noise. To address this, we propose Stochastic Gradient Lattice Random Walk (SGLRW), an extension of the Lattice Random Walk discretization. Unlike conventional Stochastic Gradient Langevin Dynamics (SGLD), SGLRW introduces stochastic noise only through the off-diagonal elements of the update covariance; this yields greater robustness to minibatch size while retaining asymptotic correctness. Furthermore, as comparison we analyze a natural analogue of SGLD utilizing gradient clipping. Experimental validation on Bayesian regression and classification demonstrates that SGLRW remains stable in regimes where SGLD fails, including in the presence of heavy-tailed gradient noise, and matches or improves predictive performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。