用混合对数正态建模事件间隔,结合Transformer预测标记,加速点过程推理。
Decomposable Transformer Point Processes
- 用对数正态混合建模事件间隔,满足马尔可夫性,避免依赖复杂采样
- 在历史事件基础上预测下一次事件,性能达到当前最佳
- 长时序预测效果优于专用基线,推理速度提升显著
传统标记点过程建模通常采用基于注意力(Transformer风格)的架构参数化强度函数,但其推断依赖计算成本高的薄化算法。本文提出新框架:以满足马尔可夫性的对数正态混合模型刻画事件间时间间隔,同时用Transformer架构建模标记的条件概率质量函数。该方法在给定历史序列的情况下预测下一个事件上取得当前最优性能。实验表明,不依赖薄化算法的推断方法显著优于依赖该算法的方法。此外,在具有挑战性的长时序预测任务中,本方法超越专为此任务设计的基线,且推理时间仅为后者的极小部分。
原文摘要 · Abstract (English)
The standard paradigm of modeling marked point processes is by parameterizing the intensity function using an attention-based (Transformer-style) architecture. Despite the flexibility of these methods, their inference is based on the computationally intensive thinning algorithm. In this work, we propose a framework where the advantages of the attention-based architecture are maintained and the limitation of the thinning algorithm is circumvented. The framework depends on modeling the conditional distribution of inter-event times with a mixture of log-normals satisfying a Markov property and the conditional probability mass function for the marks with a Transformer-based architecture. The proposed method attains state-of-the-art performance in predicting the next event of a sequence given its history. The experiments also reveal the efficacy of the methods that do not rely on the thinning algorithm during inference over the ones they do. Finally, we test our method on the challenging long-horizon prediction task and find that it outperforms a baseline developed specifically for tackling this task; importantly, inference requires just a fraction of time compared to the thinning-based baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。