新模型持续学习新主题，不遗忘旧主题，适合长期文本分析。
Continual Neural Topic Model
- 用可更新的全局先验实现持续学习，避免遗忘旧主题。
- 在主题质量和预测困惑度上优于动态主题模型。
- 能在线捕捉主题随时间的变化，适合长期文本追踪。
在持续学习中，目标是学习新任务而不遗忘旧知识。在主题模型中，这体现为在不遗忘已有主题的前提下学习新主题。现有方法或采用动态主题模型（DTM），基于全部语料一次性学习主题演化；或采用在线主题模型，随新数据持续更新但缺乏长期记忆。为此，我们提出持续神经主题模型（CoNTM），可在后续时间步持续学习主题模型而不会遗忘先前内容。该方法通过不断更新的全局先验分布实现。实验表明，CoNTM 在主题质量与预测困惑度上均优于动态主题模型，并能在线捕捉主题变化。分析显示，相比现有方法，CoNTM 能学习更丰富的主题并更好捕捉时间演变。
原文摘要 · Abstract (English)
In continual learning, our aim is to learn a new task without forgetting what was learned previously. In topic models, this translates to learning new topic models without forgetting previously learned topics. Previous work either considered Dynamic Topic Models (DTMs), which learn the evolution of topics based on the entire training corpus at once, or Online Topic Models, which are updated continuously based on new data but do not have long-term memory. To fill this gap, we propose the Continual Neural Topic Model (CoNTM), which continuously learns topic models at subsequent time steps without forgetting what was previously learned. This is achieved using a global prior distribution that is continuously updated. In our experiments, CoNTM consistently outperformed the dynamic topic model in terms of topic quality and predictive perplexity while being able to capture topic changes online. The analysis reveals that CoNTM can learn more diverse topics and better capture temporal changes than existing methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。