用神经主题模型实时捕捉文本中的新兴趋势，区分强弱信号。
BERTrend: Neural Topic Modeling for Emerging Trends Detection
- 基于BERT的在线主题建模，动态追踪话题演化。
- 提出新指标综合文档数量与更新频率，识别弱信号。
- 适合科学监测、品牌舆情等需要早期预警的场景。
在大规模、持续演化的文本语料中检测和跟踪新兴趋势与微弱信号，对科学文献监控、品牌声誉管理、关键基础设施监视等应用至关重要。现有方法常难以捕捉细微语境或动态追踪演变模式。BERTrend是一种新型神经主题建模方法，采用在线设置，引入新指标量化话题随时间的流行度，结合文档数量与更新频率，将话题分类为噪声、弱信号或强信号，从而标记出需进一步调查的新兴高增长话题。在两个真实世界大数据集上的实验表明，BERTrend能准确检测并追踪有意义的弱信号，同时有效过滤噪声，为大规模文本语料中的趋势监测提供全面解决方案。该方法亦可用于历史事件的回溯分析。此外，结合大语言模型可高效提升事件趋势的可解释性。
原文摘要 · Abstract (English)
Detecting and tracking emerging trends and weak signals in large, evolving text corpora is vital for applications such as monitoring scientific literature, managing brand reputation, surveilling critical infrastructure and more generally to any kind of text-based event detection. Existing solutions often fail to capture the nuanced context or dynamically track evolving patterns over time. BERTrend, a novel method, addresses these limitations using neural topic modeling in an online setting. It introduces a new metric to quantify topic popularity over time by considering both the number of documents and update frequency. This metric classifies topics as noise, weak, or strong signals, flagging emerging, rapidly growing topics for further investigation. Experimentation on two large real-world datasets demonstrates BERTrend's ability to accurately detect and track meaningful weak signals while filtering out noise, offering a comprehensive solution for monitoring emerging trends in large-scale, evolving text corpora. The method can also be used for retrospective analysis of past events. In addition, the use of Large Language Models together with BERTrend offers efficient means for the interpretability of trends of events.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。