arXiv:2510.24918cs.LGstat.ME2025-10

用神经网络动态融合附加信息,让主题模型更懂上下文。

Topic Analysis with Side Information: A Neural-Augmented LDA Approach

  • 用神经网络生成主题先验,自动学习侧信息与主题的关系
  • 在多个数据集上主题一致性、困惑度均优于传统方法
  • 适合有用户标签、元数据等辅助信息的个性化主题分析

传统主题模型如隐狄利克雷分配(LDA)广泛用于挖掘文本语料中的潜在结构,但难以整合元数据、用户属性或文档标签等辅助信息,限制了其表达能力、个性化和可解释性。为此,我们提出nnLDA,一种通过神经先验机制动态融合侧信息的神经增强概率主题模型。nnLDA将每篇文档建模为潜在主题的混合,其中主题比例的先验由条件于辅助特征的神经网络生成。这一设计使模型能够捕捉侧信息与主题分布间的复杂非线性交互,而静态狄利克雷先验无法实现。我们开发了一种随机变分期望最大化算法,联合优化神经与概率组件。在多个基准数据集上,nnLDA在主题一致性、困惑度及下游分类任务中持续优于LDA和狄利克雷-多项式回归。结果表明,在有侧信息可用的场景下,结合神经表示学习与概率主题建模具有显著优势。

原文摘要 · Abstract (English)

Traditional topic models such as Latent Dirichlet Allocation (LDA) have been widely used to uncover latent structures in text corpora, but they often struggle to integrate auxiliary information such as metadata, user attributes, or document labels. These limitations restrict their expressiveness, personalization, and interpretability. To address this, we propose nnLDA, a neural-augmented probabilistic topic model that dynamically incorporates side information through a neural prior mechanism. nnLDA models each document as a mixture of latent topics, where the prior over topic proportions is generated by a neural network conditioned on auxiliary features. This design allows the model to capture complex nonlinear interactions between side information and topic distributions that static Dirichlet priors cannot represent. We develop a stochastic variational Expectation-Maximization algorithm to jointly optimize the neural and probabilistic components. Across multiple benchmark datasets, nnLDA consistently outperforms LDA and Dirichlet-Multinomial Regression in topic coherence, perplexity, and downstream classification. These results highlight the benefits of combining neural representation learning with probabilistic topic modeling in settings where side information is available.

主题模型神经网络侧信息

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。