提升大模型对时间事件的感知能力,更好预测不规则时间序列事件
Enhancing Temporal Awareness in LLMs for Temporal Point Processes
- 将时间动态与语义上下文显式对齐,增强大模型时间推理能力
- 在多个基准数据集上显著提升事件预测准确率和时间似然度
- 适用于金融、医疗等需要精确时间建模的场景
时间点过程(TPPs)对于分析随时间演变的事件至关重要,广泛应用于金融、医疗和社会系统等领域。这些过程能够捕捉事件的不规则性与依赖关系,但将大语言模型(LLMs)应用于TPPs仍面临挑战。当前方法难以有效建模时间信息与语义上下文之间的复杂交互。为此,本文提出TPP-TAL框架,通过在输入前显式对齐时间动态与语义上下文,增强大模型的时间感知能力。该方法使模型更精准地捕捉事件间的长期依赖与上下文关联。在多个基准数据集上的实验表明,TPP-TAL在时间似然估计和事件预测精度上均有显著提升,验证了增强大模型时间意识对连续时间事件建模的重要性。代码已开源:https://github.com/chenlilil/TPP-TAL。
原文摘要 · Abstract (English)
Temporal point processes (TPPs) are crucial for analyzing events over time and are widely used in fields such as finance, healthcare, and social systems. These processes are particularly valuable for understanding how events unfold over time, accounting for their irregularity and dependencies. Despite the success of large language models (LLMs) in sequence modeling, applying them to temporal point processes remains challenging. A key issue is that current methods struggle to effectively capture the complex interaction between temporal information and semantic context, which is vital for accurate event modeling. In this context, we introduce TPP-TAL (Temporal Point Processes with Enhanced Temporal Awareness in LLMs), a novel plug-and-play framework designed to enhance temporal reasoning within LLMs. Rather than using the conventional method of simply concatenating event time and type embeddings, TPP-TAL explicitly aligns temporal dynamics with contextual semantics before feeding this information into the LLM. This alignment allows the model to better perceive temporal dependencies and long-range interactions between events and their surrounding contexts. Through comprehensive experiments on several benchmark datasets, it is shown that TPP-TAL delivers substantial improvements in temporal likelihood estimation and event prediction accuracy, highlighting the importance of enhancing temporal awareness in LLMs for continuous-time event modeling. The code is made available at https://github.com/chenlilil/TPP-TAL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。