解决神经标记时间点过程中的梯度冲突问题,提升建模精度
Preventing Conflicting Gradients in Neural Marked Temporal Point Processes
- 将事件时间与类型建模解耦,分任务独立训练
- 实验显示新方法在多个真实数据集上性能更优
- 适合需要精确建模复杂事件序列的研究者
神经标记时间点过程(MTPP)是捕捉带标签事件间复杂时间依赖性的灵活模型。这类模型同时学习事件到达时间与事件类型(标记)的预测分布。本文表明,训练MTPP可视为共享参数的双任务学习问题,常导致任务间梯度冲突——即各任务梯度方向相反。此时沿平均梯度更新会损害单个任务学习,降低整体性能。为此,我们提出新的参数化方法,实现对时间与类型任务的独立建模与训练,有效避免梯度冲突。在多个真实世界事件序列数据集上的实验验证了该框架相较于原始模型的优越性。
原文摘要 · Abstract (English)
Neural Marked Temporal Point Processes (MTPP) are flexible models to capture complex temporal inter-dependencies between labeled events. These models inherently learn two predictive distributions: one for the arrival times of events and another for the types of events, also known as marks. In this study, we demonstrate that learning a MTPP model can be framed as a two-task learning problem, where both tasks share a common set of trainable parameters that are optimized jointly. We show that this often leads to the emergence of conflicting gradients during training, where task-specific gradients are pointing in opposite directions. When such conflicts arise, following the average gradient can be detrimental to the learning of each individual tasks, resulting in overall degraded performance. To overcome this issue, we introduce novel parametrizations for neural MTPP models that allow for separate modeling and training of each task, effectively avoiding the problem of conflicting gradients. Through experiments on multiple real-world event sequence datasets, we demonstrate the benefits of our framework compared to the original model formulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。