用变分推断提升多项式树模型的密度估计能力,兼顾可解释性与不确定性量化。
Variational Polya Tree
- 采用变分推断替代传统马尔可夫链蒙特卡洛,实现高效后验计算。
- 在真实数据与图像上表现媲美顶尖深度密度估计方法。
- 适合需要可解释性与置信度评估的生成建模场景。
密度估计对生成建模至关重要,尤其在现代神经网络兴起背景下。现有方法虽能捕捉复杂数据分布,但缺乏可解释性与不确定性量化。贝叶斯非参数方法,尤其是多项式树(Polya Tree),通过精确刻画小区间函数行为,提供稳健框架。传统方法如马尔可夫链蒙特卡洛(MCMC)存在计算复杂度高、扩展性差的问题,限制其在深度学习中的应用。为此,我们提出变分多项式树(VPT)模型,采用随机变分推断计算后验分布。该模型提供灵活的非参数贝叶斯先验,可捕捉潜在密度,并兼容随机梯度优化。我们还利用联合分布似然,获得比传统均值场方法更精确的变分后验近似。在真实数据与图像上的实验表明,模型性能可媲美当前最先进的深度密度估计方法。同时验证了其在提升可解释性与不确定性量化方面的潜力。代码已公开于 https://github.com/howardchanth/var-polya-tree。
原文摘要 · Abstract (English)
Density estimation is essential for generative modeling, particularly with the rise of modern neural networks. While existing methods capture complex data distributions, they often lack interpretability and uncertainty quantification. Bayesian nonparametric methods, especially the \polya tree, offer a robust framework that addresses these issues by accurately capturing function behavior over small intervals. Traditional techniques like Markov chain Monte Carlo (MCMC) face high computational complexity and scalability limitations, hindering the use of Bayesian nonparametric methods in deep learning. To tackle this, we introduce the variational \polya tree (VPT) model, which employs stochastic variational inference to compute posterior distributions. This model provides a flexible, nonparametric Bayesian prior that captures latent densities and works well with stochastic gradient optimization. We also leverage the joint distribution likelihood for a more precise variational posterior approximation than traditional mean-field methods. We evaluate the model performance on both real data and images, and demonstrate its competitiveness with other state-of-the-art deep density estimation methods. We also explore its ability in enhancing interpretability and uncertainty quantification. Code is available at https://github.com/howardchanth/var-polya-tree.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。