通过动态标签平均提升时间图神经网络训练效率
Never Skip a Batch: Dense Learning of Temporal GNNs via Adaptive Pseudo-Supervision
- 用历史标签构建软伪标签，填补无监督批次空白
- 在多个基准数据集上实现6倍更快达到最优性能
- 无需修改模型结构，通用性强，适合实际动态图场景
时间图网络在真实动态图中面临监督信号不规则的问题，多数小批量样本包含极少标记事件。标签缺失导致梯度更新方差大，进而造成收敛缓慢。为此，我们提出移动平均标签（MAL），基于历史监督信号生成软伪目标，维持模型和损失函数不变。该方法将监督空缺替换为独立于时间图模型及消息传递或记忆组件的有意义信号。理论分析表明，在温和假设下，聚合历史监督信息可降低随机梯度方差，加速收敛。实验显示，对TGNv2和DyRepv2（我们改进的DyRep）模型，MAL显著提升预测性能，达成新SOTA，且平均提速6倍即可达到最高得分，适用于一系列时间图基准数据集。
原文摘要 · Abstract (English)
Temporal graph networks suffer from irregular supervision in realworld dynamic graphs, as most minibatches contain few labeled events. The lack of labels leads to high-variance gradient updates and, consequently, slow wall-clock convergence. To constructively reduce sparsity, our Moving-Averaged Labels (MAL) assigns soft pseudo-targets based on past supervised signals using a running label distribution while leaving the loss and the model architecture unchanged. Thus, supervision gaps are replaced with informative signals independent of a temporal graph model and the message passing or memory components used. Theoretical analysis supports our insight that aggregating historical supervision into moving average targets reduces stochastic gradient variance, yielding faster convergence under mild assumptions. Experimentally, for TGNv2 and DyRepv2 (our modification of DyRep) models, MAL boosts predictive performance, establishing a new SOTA, and improves time-to-accuracy (on average 6x faster to reach the top score) for a common suite of Temporal Graph Benchmark datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。