用时间衰减与注意力机制追踪话题随时间演变,提升主题模型的连贯性与可解释性。
Dynamic Topic Evolution with Temporal Decay and Attention in Large Language Models
- 引入时间衰减函数与注意力机制,动态调整语义单元权重。
- 通过状态转移矩阵建模话题演化,实现生成多样性与平滑性的平衡。
- 适用于需分析长期文本趋势的场景,如舆情监控与知识演进研究。
本文提出一种基于时序大语言模型的动态话题演化建模框架。首先利用大语言模型获取文本上下文嵌入,再引入时间衰减函数与注意力机制,根据时间间隔调整语义单元重要性,捕捉不同时期的话题变化。时序表示被映射至潜在话题空间,并应用状态转移矩阵描述话题的动态演化。联合优化目标同时约束语义建模与时间一致性,确保话题生成的多样性和平滑性。该设计统一建模语义表征与时间演化,提升了话题连贯性、多样性及时间稳定性与可解释性。在真实语料上的实验表明,该框架能有效捕捉话题的产生、扩展与衰退,且在多项指标上优于现有模型。整体方法为理解大规模文本中的动态语义模式提供系统性解决方案,丰富了话题建模的研究范式,支持多领域复杂文本分析任务。
原文摘要 · Abstract (English)
This paper proposes a modeling framework for dynamic topic evolution based on temporal large language models. The method first uses a large language model to obtain contextual embeddings of text and then introduces a temporal decay function and an attention mechanism. These components allow the model to adjust the importance of semantic units according to time intervals and capture topic variations across different periods. The temporal representations are then mapped into a latent topic space, where a state transition matrix is applied to describe the dynamic evolution of topics. A joint optimization objective constrains both semantic modeling and temporal consistency, ensuring diversity and smoothness in topic generation. The design emphasizes the unified modeling of semantic representation and temporal evolution, which improves topic coherence and diversity while enhancing stability and interpretability over time. Experiments on real-world corpora show that the framework effectively captures the generation, expansion, and decline of topics and outperforms existing models across multiple metrics. Overall, the proposed method provides a systematic solution for understanding dynamic semantic patterns in large-scale text, enriches the research paradigm of topic modeling, and supports complex text analysis tasks in multiple domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。