arXiv:2409.14500cs.LGcs.AI2024-09NeurIPS被引 18

构建14个工业图数据集基准,评估图模型在真实场景下的表现。

GraphLand: Evaluating Graph Machine Learning Models on Diverse Industrial Data

  • 构建GraphLand基准,覆盖14个不同工业领域的图数据。
  • 发现现有图基础模型在新数据上表现不佳,远低于预期。
  • 验证梯度提升树+图特征是强基线,适合工业场景对比。

尽管图结构数据在多个工业领域广泛应用,但主流图机器学习基准在节点属性预测任务中仅覆盖极少数数据领域,且图神经网络常在少量学术引用网络上评估。这一问题在图基础模型兴起背景下尤为突出:这些模型本应具备跨域迁移能力,却普遍只在有限的窄域数据集上测试。为缓解此问题,我们提出GraphLand:一个包含14个来自不同工业应用的节点属性预测图数据集的基准。GraphLand支持在统一框架下评估模型在多样规模、结构和特征的图上的表现,并可研究此前未充分探索的问题,如在归纳与直推设置下真实时间分布偏移对模型性能的影响。为模拟真实工业环境,我们使用GraphLand对比了图神经网络与工业中流行的梯度提升决策树(GBDT)模型,发现提供额外图特征的GBDT可成为非常强劲的基线。进一步评估当前通用图基础模型,发现其在新数据集上无法产生有竞争力的结果。

原文摘要 · Abstract (English)

Although data that can be naturally represented as graphs is widespread in real-world applications across diverse industries, popular graph ML benchmarks for node property prediction only cover a surprisingly narrow set of data domains, and graph neural networks (GNNs) are often evaluated on just a few academic citation networks. This issue is particularly pressing in light of the recent growing interest in designing graph foundation models. These models are supposed to be able to transfer to diverse graph datasets from different domains, and yet the proposed graph foundation models are often evaluated on a very limited set of datasets from narrow applications. To alleviate this issue, we introduce GraphLand: a benchmark of 14 diverse graph datasets for node property prediction from a range of different industrial applications. GraphLand allows evaluating graph ML models on a wide range of graphs with diverse sizes, structural characteristics, and feature sets, all in a unified setting. Further, GraphLand allows investigating such previously underexplored research questions as how realistic temporal distributional shifts under transductive and inductive settings influence graph ML model performance. To mimic realistic industrial settings, we use GraphLand to compare GNNs with gradient-boosted decision trees (GBDT) models that are popular in industrial applications and show that GBDTs provided with additional graph-based input features can sometimes be very strong baselines. Further, we evaluate currently available general-purpose graph foundation models and find that they fail to produce competitive results on our proposed datasets.

图神经网络基准测试工业应用迁移能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。