用上下文学习让模型直接推断事件序列规律，无需重新训练
In-Context Learning of Temporal Point Processes with Foundation Inference Models
- 预训练大模型，通过上下文直接推断事件发生概率
- 在多个真实数据集上表现接近专用模型
- 适合快速适配新场景，无需额外调参
多类型标记时间点过程（MTPPs）为建模多类型事件序列提供了一种系统化方法，可用于揭示动态规律并预测未来事件。现有神经网络方法需为每个目标系统单独训练专用模型。本文提出全新思路：基于摊销推断与上下文学习，预训练一个深度神经网络，使其能从一组事件序列构成的上下文中，推断出事件历史的条件强度函数。预训练使用大规模合成数据，采样自广义霍克斯过程分布。模型预训练完成后，可直接用于真实数据的MTPP推断，无需额外训练，或仅需少量微调即可适配特定系统。实验表明，该摊销方法在多个基准数据集上的未来事件预测性能，可媲美专用模型。
原文摘要 · Abstract (English)
Modeling event sequences of multiple event types with marked temporal point processes (MTPPs) provides a principled way to uncover governing dynamical rules and predict future events. Current neural network approaches to MTPP inference rely on training separate, specialized models for each target system. We pursue a radically different approach: drawing on amortized inference and in-context learning, we pretrain a deep neural network to infer, in-context, the conditional intensity functions of event histories from a context defined by sets of event sequences. Pretraining is performed on a large synthetic dataset of MTPPs sampled from a broad distribution of Hawkes processes. Once pretrained, our Foundation Inference Model for Point Processes (FIM-PP) can estimate MTPPs from real-world data without any additional training, or be rapidly finetuned to target systems. Experiments show that this amortized approach matches the performance of specialized models on next-event prediction across common benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。