不依赖大模型也不需调参,用统一框架实现图数据的少样本快速学习。
GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning
- 基于令牌的统一框架,支持节点、边、图多层级任务
- 在少样本场景下性能超越基线,训练耗时大幅减少
- 适合处理异构图数据,特别适合资源有限的场景
图神经网络在处理关系型数据方面表现强劲,但难以泛化到未见过的图结构,催生了图基础模型(GFMs)的发展。然而,当前GFMs面临图数据极端异质性的挑战:每张图可能拥有独特的特征空间、标签集和拓扑结构。现有方法主要分为两类:一是依赖大语言模型(LLM),但受限于文本输入,难以处理大规模数值型图数据;二是预训练结构化模型,但新任务适配通常需要代价高昂的逐图调参,效率低下。本文提出无需大模型、无需调参的图上下文学习变压器(GILT),通过新型令牌机制,在统一框架内实现节点、边、图级别的分类任务。该机制可处理通用数值特征,并动态从上下文中理解类别语义,实现无调参自适应。大量实验表明,GILT在少样本场景下性能更优,且耗时显著低于基于LLM或需调参的基线方法。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) are powerful tools for processing relational data but often struggle to generalize to unseen graphs, giving rise to the development of Graph Foundational Models (GFMs). However, current GFMs are challenged by the extreme heterogeneity of graph data, where each graph can possess a unique feature space, label set, and topology. To address this, two main paradigms have emerged. The first leverages Large Language Models (LLMs), but is fundamentally text-dependent, thus struggles to handle the numerical features in vast graphs. The second pre-trains a structure-based model, but the adaptation to new tasks typically requires a costly, per-graph tuning stage, creating a critical efficiency bottleneck. In this work, we move beyond these limitations and introduce \textbf{G}raph \textbf{I}n-context \textbf{L}earning \textbf{T}ransformer (GILT), a framework built on an LLM-free and tuning-free architecture. GILT introduces a novel token-based framework for in-context learning (ICL) on graphs, reframing classification tasks spanning node, edge and graph levels in a unified framework. This mechanism is the key to handling heterogeneity, as it is designed to operate on generic numerical features. Further, its ability to understand class semantics dynamically from the context enables tuning-free adaptation. Comprehensive experiments show that GILT achieves stronger few-shot performance with significantly less time than LLM-based or tuning-based baselines, validating the effectiveness of our approach. Our code is available at: https://github.com/yiming421/inductnode/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。