用霍克斯过程思想改进注意力机制，更好捕捉事件时间与类型关系。
From Hawkes Processes to Attention: Time-Modulated Mechanisms for Event Sequences
- 基于多变量霍克斯过程设计可学习的时序调制注意力
- 在多个数据集上优于传统Transformer方法，提升事件预测精度
- 适合处理医疗、金融等具有复杂时间依赖的事件序列
标记时间点过程（MTPPs）广泛存在于医疗、社交、商业和金融等领域。现有基于Transformer的方法通常仅通过位置编码引入时间信息，依赖共享或参数化衰减结构，难以捕捉异质且类型特异的时间效应。受此启发，本文从多变量霍克斯过程理论出发，推导出一种新型注意力算子——霍克斯注意力（Hawkes Attention），利用每类事件的可学习神经核函数调制查询、键和值的投影，从而替代传统注意力中的对应部分。该设计使事件时间与内容交互统一建模，能够从数据中学习时间相关行为及类型特异性激发模式。实验表明，该方法在多个基准上表现更优。此外，该注意力机制还可灵活应用于时间序列预测等特定时序结构任务。
原文摘要 · Abstract (English)
Marked Temporal Point Processes (MTPPs) arise naturally in medical, social, commercial, and financial domains. However, existing Transformer-based methods mostly inject temporal information only via positional encodings, relying on shared or parametric decay structures, which limits their ability to capture heterogeneous and type-specific temporal effects. Inspired by this observation, we derive a novel attention operator called Hawkes Attention from the multivariate Hawkes process theory for MTPP, using learnable per-type neural kernels to modulate query, key and value projections, thereby replacing the corresponding parts in the traditional attention. Benefited from the design, Hawkes Attention unifies event timing and content interaction, learning both the time-relevant behavior and type-specific excitation patterns from the data. The experimental results show that our method achieves better performance compared to the baselines. In addition to the general MTPP, our attention mechanism can also be easily applied to specific temporal structures, such as time series forecasting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。