arXiv:2410.02296cs.CL2024-10Conference of the …被引 5

用两种增强策略让通用语言模型媲美专业图神经网络的节点分类能力。

How to Make LMs Strong Node Classifiers?

  • 输入端融合拓扑与语义信息,输出端用轻量GNN筛选类别候选。
  • Flan-T5在真实数据集上达到与顶尖图模型相当的准确率。
  • 无需修改模型架构,适合多任务联合训练场景。

语言模型(LMs)正逐步挑战图神经网络(GNNs)和图变换器(GTs)在图学习任务中的主导地位。本文提出一种新方法,使现成的通用语言模型在不改变原有结构的前提下,实现与最先进(SOTA)GNNs相当的节点分类性能。该方法保留了语言模型指令微调的核心优势:可跨多种数据集联合训练,提升灵活性与效率。具体通过两项关键增强策略实现:(1) 利用拓扑与语义检索方法丰富语言模型的输入,提供更丰富的上下文信息;(2) 通过轻量级GNN分类器引导语言模型的分类过程,有效剪枝类别候选。在真实世界数据集上的实验表明,采用这些增强策略的Flan-T5骨干模型,在节点分类任务中优于现有最先进的文本输出节点分类器,并可与表现最佳的向量输出节点分类器相媲美。本工作弥合了专用节点分类器与通用语言模型之间的差距,为更通用、更广泛适用的图学习模型铺平道路。代码将在发表后开源。

原文摘要 · Abstract (English)

Language Models (LMs) are increasingly challenging the dominance of domain-specific models, such as Graph Neural Networks (GNNs) and Graph Transformers (GTs), in graph learning tasks. Following this trend, we propose a novel approach that empowers off-the-shelf LMs to achieve performance comparable to state-of-the-art (SOTA) GNNs on node classification tasks, without requiring any architectural modification. By preserving the LM's original architecture, our approach retains a key benefit of LM instruction tuning: the ability to jointly train on diverse datasets, fostering greater flexibility and efficiency. To achieve this, we introduce two key augmentation strategies: (1) Enriching LMs' input using topological and semantic retrieval methods, which provide richer contextual information, and (2) guiding the LMs' classification process through a lightweight GNN classifier that effectively prunes class candidates. Our experiments on real-world datasets show that backbone Flan-T5 LMs equipped with these augmentation strategies outperform SOTA text-output node classifiers and are comparable to top-performing vector-output node classifiers. By bridging the gap between specialized node classifiers and general LMs, this work paves the way for more versatile and widely applicable graph learning models. We will open-source the code upon publication.

语言模型图学习节点分类增强策略

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。