提出新方法让图神经网络在大批次训练时仍能保持时间连续性。
A Batch-Insensitive Dynamic GNN Approach to Address Temporal Discontinuity in Graph Streams
- 通过时间李普希茨正则化控制参数搜索空间扩张。
- 在三个数据集上支持更大批次，训练速度更快且性能稳定。
- 适合需要高效训练动态图模型的研究者与工业应用。
在动态图中，保持时间连续性至关重要。然而，使用大批次训练的基于记忆的动态图神经网络（MDGNNs）常会破坏事件序列，导致时间信息丢失。这种不连续性不仅损害时间建模效果，还因增加参数收敛难度而阻碍优化。我们的理论研究表明，大批次会扩大参数搜索空间，其上限可通过李普希茨界量化。为此，我们提出BADGNN，一个批处理无关框架，包含两个核心组件：(1) 时间李普希茨正则化（TLR），用于控制参数搜索空间扩展；(2) 自适应注意力调整（A3），缓解正则化和批处理带来的注意力失真。在三个基准数据集上的实证结果表明，与TGN相比，BADGNN在支持显著更大的批量的同时保持强劲性能并实现更快训练速度。代码已公开于https://anonymous.4open.science/r/TGN_Lipichitz-C033/。
原文摘要 · Abstract (English)
In dynamic graphs, preserving temporal continuity is critical. However, Memory-based Dynamic Graph Neural Networks (MDGNNs) trained with large batches often disrupt event sequences, leading to temporal information loss. This discontinuity not only deteriorates temporal modeling but also hinders optimization by increasing the difficulty of parameter convergence. Our theoretical study quantifies this through a Lipschitz upper bound, showing that large batch sizes enlarge the parameter search space. In response, we propose BADGNN, a novel batch-agnostic framework consisting of two core components: (1) Temporal Lipschitz Regularization (TLR) to control parameter search space expansion, and (2) Adaptive Attention Adjustment (A3) to alleviate attention distortion induced by both regularization and batching. Empirical results on three benchmark datasets show that BADGNN maintains strong performance while enabling significantly larger batch sizes and faster training compared to TGN. Our code is available at Code: https://anonymous.4open.science/r/TGN_Lipichitz-C033/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。