arXiv:2607.24583cs.LG2026-07

提出新方法实现大规模非共轭后验的高效贝叶斯学习,无需闭式解。

PYPM-GGD: Pitman-Yor Process Mixture with Generalized Gaussian Density using ADAM

论文配图:PYPM-GGD: Pitman-Yor Process Mixture with Generalized Gaussian Density using ADAM
图 1 · 摘自论文原文
  • 采用自适应步长的随机梯度上升,避免对变分后验的闭式表达要求。
  • 在MIT67和SUN397数据集上达到或超过现有先进聚类方法性能。
  • 适用于复杂模型如ResNet特征,适合高类别数的大规模数据场景。

大规模贝叶斯非参数学习器(如随机变分推断,SVI)可实现大类别数、大规模训练下的分数级成本计算。与前代方法类似,SVI依赖共轭变分后验假设来近似真实后验。然而,处理非共轭后验的大规模学习更具挑战性。近期工作多采用蒙特卡洛方法近似学习器,但通常仅在逻辑回归等简单模型上验证,因计算开销较高。为克服SVI局限,本文提出基于常步长随机梯度上升的新方法,实现非共轭后验的大规模学习。与SVI不同,该方法不要求变分后验期望的闭式表达,仅需后验可微。为确保随机环境下收敛,传统SVI采用衰减步长,我们受SVI和Adam启发,引入自适应步长显著提升学习效率。实验表明,该方法兼容ResNet特征,在MIT67和SUN397等高类别数数据集上表现优异,并在聚类指标上达到或超越最新方法。

原文摘要 · Abstract (English)

Large scale Bayesian nonparametrics (BNP) learner such as Stochastic Variational Inference (SVI) can handle datasets with large class number and large training size at fractional cost. Like its predecessor, SVI rely on the assumption of conjugate variational posterior to approximate the true posterior. A more challenging problem is to consider large scale learning on non-conjugate posterior. Recent works in this direction are mostly associated with using Monte Carlo methods for approximating the learner. However, these works are usually demonstrated on non-BNP related task and less complex models such as logistic regression, due to higher computational complexity. In order to overcome the issue faced by SVI, we develop a novel approach based on the recently proposed constant stepsize stochastic gradient ascent to allow large scale learning on non-conjugate posterior. Unlike SVI, our new learner does not require closed- form expression for the variational posterior expectatations. Our only requirement is that the variational posterior is differentiable. In order to ensure convergence in stochastic settings, SVI rely on decaying step-sizes to slow its learning. Inspired by SVI and Adam, we propose the novel use of adaptive stepsizes in our method to significantly improve its learning. We show that our proposed methods is compatible with ResNet features when applied to large class number datasets such as MIT67 and SUN397. Finally, we compare our proposed learner with several recent works such as deep clustering algorithms and showed we were able to produce on-par or outperform the state-of-the-art methods in terms of clustering measures.

贝叶斯学习非共轭后验自适应优化聚类

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。