用向量拷贝模型提升复杂模型后验近似的准确性。
Vector Copula Variational Inference and Dependent Block Posterior Approximations
- 用可学习的传输映射构建分块边缘分布,通过向量拷贝捕捉块间依赖。
- 在16个数据集上优于独立分块和因子依赖的基准方法,计算开销小。
- 适合需要高精度后验近似的大模型,如稀疏正则化与层次时间序列模型。
变分推断的关键在于选择可处理的密度来近似贝叶斯后验。对于大型复杂模型,常见做法是假设参数空间划分后的多维块之间相互独立,虽简化问题但可能降低精度。本文提出使用向量拷贝模型,以简洁方式捕捉块间依赖关系。通过可学习的传输映射构造定制化的多变量边缘分布,得到称为“依赖分块后验”的联合分布近似。提出的向量拷贝模型支持灵活的变分近似,允许不同边缘分布、块数、块大小及块间依赖形式,并能使用高效的随机梯度优化求解。该方法在四个统计模型和16个具有挑战性的数据集上进行了验证,包括使用全局-局部收缩先验的正则化模型,以及用于平滑和异方差时间序列的层次模型。所有情况下,该方法均在有限额外计算成本下,比假设块独立或基于因子依赖的基准方法产生更准确的后验近似。相关Python包已开源:https://github.com/YuFuOliver/VCVI_Rep_PyPackage。
原文摘要 · Abstract (English)
The key to VI is the selection of a tractable density to approximate the Bayesian posterior. For large and complex models a common choice is to assume independence between multivariate blocks in a partition of the parameter space. While this simplifies the problem it can reduce accuracy. This paper proposes using vector copulas to capture dependence between the blocks parsimoniously. Tailored multivariate marginals are constructed using learnable transport maps. We call the resulting joint distribution a ``dependent block posterior'' approximation. Vector copula models are suggested that make tractable and flexible variational approximations. They allow for differing marginals, numbers of blocks, block sizes and forms of between block dependence. They also allow for solution of the variational optimization using efficient stochastic gradient methods. The approach is demonstrated using four different statistical models and 16 datasets which have posteriors that are challenging to approximate. This includes models that use global-local shrinkage priors for regularization, and hierarchical models for smoothing and heteroscedastic time series. In all cases, our method produces more accurate posterior approximations than benchmark VI methods that either assume block independence or factor-based dependence, at limited additional computational cost. A python package implementing the method is available on GitHub at https://github.com/YuFuOliver/VCVI_Rep_PyPackage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。