用更灵活的贝塔-刘维尔先验改进短文本聚类模型
Hierarchical mixtures of Unigram models for short text clustering: The role of Beta-Liouville priors
- 以贝塔-刘维尔分布替代传统狄利克雷先验，增强文本特征相关性建模能力
- 推导出可变分推断的更新公式，支持对短文本聚类参数的近似后验估计
- 提出随机化变分算法提升大规模数据处理效率，适合短文本分析场景
本文提出一种针对短文本无监督分类的多项式混合模型变体。传统模型中多项式概率向量采用狄利克雷先验，本文则探索基于贝塔-刘维尔分布的替代方案，其相关结构更具灵活性。研究重点分析了贝塔-刘维尔分布的理论性质，尤其关注其与多项式似然的共轭性。该性质使我们能够推导出坐标上升变分推断（CAVI）算法的参数更新方程，从而实现模型参数的近似后验推断。此外，引入一种随机化版CAVI算法以提升可扩展性。论文最后通过实证示例展示了贝塔-刘维尔超参数的有效选择策略。
原文摘要 · Abstract (English)
This paper presents a variant of the Multinomial mixture model tailored to the unsupervised classification of short text data. While the Multinomial probability vector is traditionally assigned a Dirichlet prior distribution, this work explores an alternative formulation based on the Beta-Liouville distribution, which offers a more flexible correlation structure than the Dirichlet. We examine the theoretical properties of the Beta-Liouville distribution, with particular focus on its conjugacy with the Multinomial likelihood. This property enables the derivation of update equations for a CAVI (Coordinate Ascent Variational Inference) algorithm, facilitating approximate posterior inference of the model parameters. In addition, we introduce a stochastic variant of the CAVI algorithm to enhance scalability. The paper concludes with empirical examples demonstrating effective strategies for selecting the Beta-Liouville hyperparameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。