新基准挑战时序图网络,逼其学会复杂动态序列行为
TGB-Seq Benchmark: Challenging Temporal GNNs with Complex Sequential Dynamics
- 设计去重数据集,强制模型学习真实世界的序列依赖
- 现有方法在新数据集上性能暴跌,训练成本翻倍
- 适合研究动态推荐、社交网络预测的学者使用
未来链接预测是各类动态系统中的核心挑战。尽管已有众多时序图神经网络(temporal GNNs)和基准数据集,但这些数据集普遍存在大量重复边,缺乏真实场景中的复杂序列动态特征,如推荐系统和社交网络中‘关注了OpenAI和Anthropic的人更可能关注Meta的AI’这类行为模式。本文揭示,现有方法如GraphMixer和DyGFormer本质上无法学习此类简单序列动态。为此,我们提出时间图基准序列动态版(TGB-Seq),通过精心筛选真实世界数据,包括电商交互、电影评分、商业评论、社交网络、引用网络和网页链接网络等,显著减少重复边,迫使模型真正学习序列演化规律。基准测试显示,当前方法在TGB-Seq上普遍出现性能大幅下降,并伴随高昂训练成本,为未来研究带来新挑战与机遇。TGB-Seq数据集、排行榜及示例代码已公开于https://tgb-seq.github.io/。
原文摘要 · Abstract (English)
Future link prediction is a fundamental challenge in various real-world dynamic systems. To address this, numerous temporal graph neural networks (temporal GNNs) and benchmark datasets have been developed. However, these datasets often feature excessive repeated edges and lack complex sequential dynamics, a key characteristic inherent in many real-world applications such as recommender systems and ``Who-To-Follow'' on social networks. This oversight has led existing methods to inadvertently downplay the importance of learning sequential dynamics, focusing primarily on predicting repeated edges. In this study, we demonstrate that existing methods, such as GraphMixer and DyGFormer, are inherently incapable of learning simple sequential dynamics, such as ``a user who has followed OpenAI and Anthropic is more likely to follow AI at Meta next.'' Motivated by this issue, we introduce the Temporal Graph Benchmark with Sequential Dynamics (TGB-Seq), a new benchmark carefully curated to minimize repeated edges, challenging models to learn sequential dynamics and generalize to unseen edges. TGB-Seq comprises large real-world datasets spanning diverse domains, including e-commerce interactions, movie ratings, business reviews, social networks, citation networks and web link networks. Benchmarking experiments reveal that current methods usually suffer significant performance degradation and incur substantial training costs on TGB-Seq, posing new challenges and opportunities for future research. TGB-Seq datasets, leaderboards, and example codes are available at https://tgb-seq.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。