用多元高斯分布建模主题,提升短文本主题可解释性。
Multivariate Gaussian Topic Modelling: A novel approach to discover topics with greater semantic coherence
- 将主题表示为多元高斯分布,文档建模为高斯混合模型。
- 在20个新闻组数据集上实现0.7的平均主题连贯性,优于基准模型。
- 适合需要高可解释性的主题分析场景,如舆情监控、知识发现。
文本挖掘中,主题建模用于从文档中发现语义主题。传统生成式模型如LDA或LSA虽能识别潜在主题,但存在可解释性差和短文本表现不佳的问题。本文提出一种新的多元高斯主题模型(MGTM),将主题表示为多元高斯分布,文档建模为高斯混合模型。通过在文档语料上应用EM算法,识别出对应于潜在主题的多元高斯分布及其参数。分析各分布参数可提取主题关键词,并据此进行主题标注。该方法在20个新闻组数据集上与4种基准模型对比,结果显示其平均主题连贯性达0.7,中位数为0.76,显著高于其他模型,证明其在捕捉语义主题并提升可解释性方面的有效性。
原文摘要 · Abstract (English)
An important aspect of text mining involves information retrieval in form of discovery of semantic themes (topics) from documents using topic modelling. While generative topic models like Latent Dirichlet Allocation (LDA) or Latent Semantic Analysis (LSA) elegantly model topics as probability distributions and are useful in identifying latent topics from large document corpora with minimal supervision, they suffer from difficulty in topic interpretability and reduced performance in shorter texts. Here we propose a novel Multivariate Gaussian Topic Model (MGTM). In this approach topics are presented as Multivariate Gaussian Distributions and documents as Gaussian Mixture Models. Applying EM algorithm on a document corpus, the various constituent Multivariate Gaussian distributions corresponding to the latent topics and their respective parameters are identified. Analysis of the parameters of each distribution helps identify the respective topic keywords, and from these key-words topic annotations are carried out. This approach is applied on 20 newsgroups dataset to demonstrate the interpretability benefits vis-`a-vis 4 other benchmark models. The effectiveness of this model in capturing the semantic theme of the topics with high interpretability is examined by calculating the topic coherence and comparing the coherence values with benchmark models. This model achieves a highest mean topic coherence (0.7) and median topic coherence (0.76) vis-`a-vis the benchmark models, demonstrating high effectiveness in identifying interpretable, semantically coherent topics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。