通过自相似性筛选,让主题模型更准地选出关键词。
CAST: Corpus-Aware Self-similarity Enhanced Topic modelling
- 用语料上下文增强候选关键词嵌入,提升语义准确性。
- 利用词嵌入自相似性过滤功能词,避免干扰主题生成。
- 在新闻和推特数据上表现优异,适合处理噪声数据的主题建模。
主题建模是挖掘大规模文档集合中潜在信息的关键无监督学习技术。现有神经主题建模方法常关注文档的上下文信息,却忽略了候选中心词的上下文细节,导致主题词选择不准确。同时,功能词常被误选为主题词。为此,我们提出CAST:一种基于语料感知的自相似性增强主题建模方法。该方法使用数据集上下文化的候选中心词嵌入,并引入基于自相似性的新方法过滤无意义词项。受对比学习启发,我们发现功能词在不同上下文中的自相似性远低于主题词,因此自相似性可有效防止功能词成为候选主题词。实验表明,该方法显著提升了主题的连贯性与多样性,增强了对噪声数据的鲁棒性。在新闻基准数据集和一个推特数据集上的实验验证了其优于强基线的方法性能。
原文摘要 · Abstract (English)
Topic modelling is a pivotal unsupervised machine learning technique for extracting valuable insights from large document collections. Existing neural topic modelling methods often encode contextual information of documents, while ignoring contextual details of candidate centroid words, leading to the inaccurate selection of topic words due to the contextualization gap. In parallel, it is found that functional words are frequently selected over topical words. To address these limitations, we introduce CAST: Corpus-Aware Self-similarity Enhanced Topic modelling, a novel topic modelling method that builds upon candidate centroid word embeddings contextualized on the dataset, and a novel self-similarity-based method to filter out less meaningful tokens. Inspired by findings in contrastive learning that self-similarities of functional token embeddings in different contexts are much lower than topical tokens, we find self-similarity to be an effective metric to prevent functional words from acting as candidate topic words. Our approach significantly enhances the coherence and diversity of generated topics, as well as the topic model's ability to handle noisy data. Experiments on news benchmark datasets and one Twitter dataset demonstrate the method's superiority in generating coherent, diverse topics, and handling noisy data, outperforming strong baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。