arXiv:2511.02345cs.LGstat.CO2025-11

用简化版归一化流提升马尔可夫链采样效率,特别适合小样本复杂模型。

Reducing normalizing flow complexity for MCMC preconditioning

  • 将线性预处理与条件归一化流结合,按数据维度特性分治处理
  • 在合成数据和稀疏逻辑回归上显著改善尾部采样质量,有效样本量更高
  • 自动适应目标分布几何,适合小样本高阶贝叶斯建模场景

预处理是提升马尔可夫链蒙特卡洛(MCMC)采样效率的关键,通过可逆映射帮助探索几何复杂的后验分布。尽管线性预处理对中等复杂度分布已足够,但近期研究尝试用归一化流(NF)构建非线性预处理,然而实证与理论表明过参数化的NF会降低采样效率与拟合质量。现有方法未根据目标分布自适应调整架构。我们提出一种因子分解式预处理结构:将线性组件应用于近似高斯的维度(由预热样本估计),用条件归一化流处理更复杂的维度。该方法在两个复杂合成分布上显著改善尾部样本,在不同似然与先验强度下的稀疏逻辑回归后验中表现更优,并在弱似然、强漏斗几何的层次贝叶斯模型中获得更高有效样本数。适用于数据有限的层次贝叶斯分析,可为神经MCMC设计的理论与工具发展提供参考。

原文摘要 · Abstract (English)

Preconditioning is a key component of MCMC algorithms that improves sampling efficiency by facilitating exploration of geometrically complex target distributions through an invertible map. While linear preconditioners are often sufficient for moderately complex target distributions, recent work has explored nonlinear preconditioning with invertible neural networks as components of normalizing flows (NFs). However, empirical and theoretical studies show that overparameterized NF preconditioners can degrade sampling efficiency and fit quality. Moreover, existing NF-based approaches do not adapt their architectures to the target distribution. Related work outside of MCMC similarly finds that suitably parameterized NFs can achieve comparable or superior performance with substantially less training time or data. We propose a factorized preconditioning architecture that reduces NF complexity by combining a linear component with a conditional NF, improving adaptability to target geometry. The linear preconditioner is applied to dimensions that are approximately Gaussian, as estimated from warmup samples, while the conditional NF models more complex dimensions. Our method yields significantly better tail samples on two complex synthetic distributions and consistently better performance on a sparse logistic regression posterior across varying likelihood and prior strengths. It also achieves higher effective sample sizes on hierarchical Bayesian model posteriors with weak likelihoods and strong funnel geometries. This approach is particularly relevant for hierarchical Bayesian model analyses with limited data and could inform current theoretical and software strides in neural MCMC design.

MCMC贝叶斯推断归一化流采样效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。