提出无需参数的时序编码方法,让知识图谱模型跨数据集泛化。
GRATE: Temporal Extensions for Inductive KG Foundation Models via Gated Rotary Attention
- 用旋转注意力和查询门控编码时间差,不增加可学习参数。
- 在跨数据集测试中,单一预训练模型在多数场景优于静态基线。
- 构建新基准测试跨时间、实体、关系的归纳迁移能力,适合研究者参考。
知识图谱基础模型如Ultra和Trix通过学习关系图表示,实现了对未见实体和关系的强大归纳迁移能力。将这种迁移性扩展到时序知识图谱(TKG)仍具挑战:现有时序模型的参数绑定于特定数据集的实体、关系或时间戳,无法适应词汇不重叠的TKG。我们提出GRATE(Gated Rotary Attention for Temporal Encoding),一种无需可学习参数的边端消息函数,通过相对时间差旋转每条边消息,并引入查询条件门控筛选时序相关信号。GRATE可集成至类似NBFNet的KG基础模型,同时保持结构迁移能力。现有TKG基准测试均在共享训练/测试词汇下进行,无法直接评估跨数据集时序迁移;为此,我们构建了GDELTIndT和WIKIIndT两个归纳迁移基准套件,包含分离的实体、关系与时间戳,涵盖插值与外推场景。在这些基准及保留预测数据集上,单个联合预训练的GRATE检查点在多数设置中优于静态基线模型。
原文摘要 · Abstract (English)
Knowledge graph foundation models such as Ultra and Trix achieve strong inductive transfer by learning relation-graph representations that generalise to unseen entities and relations. Extending this transferability to temporal knowledge graphs (TKGs) remains challenging: existing temporal models tie their parameters to dataset-specific entities, relations, or timestamps and are not designed to transfer to TKGs with disjoint vocabularies. We propose GRATE (Gated Rotary Attention for Temporal Encoding), an entity-side message function that adds no learnable parameters and encodes time through relative time differences by rotating each edge message according to its time gap to the query and applying a query-conditioned gate to select temporally relevant signals. GRATE integrates into NBFNet-style KG foundation models while preserving structural transferability. Existing TKG benchmarks evaluate within shared train/test vocabularies and cannot directly test cross-dataset temporal transfer; we therefore construct GDELTIndT and WIKIIndT, inductive transfer benchmark suites with disjoint entities, relations, and timestamps spanning both interpolation and extrapolation. Across these benchmarks and held-out forecasting datasets, a single jointly pretrained GRATE checkpoint improves over the static base model in most settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。