用GNN当裁判,让LLM在少样本图学习中更准
GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback

- 让GNN和LLM协作生成可信伪标签
- 在低资源下准确率超越现有方法
- 适合小样本图数据标注场景
大型语言模型(LLMs)因对文本节点特征的语义理解能力强,在文本属性图(TAGs)上表现优异。但在标签节点严重稀缺的低资源场景下,其作为预测器的效果受限,因微调LLM通常需要充足标注数据,尤其当图结构复杂时。本文针对两大挑战:(i)在TAG上生成与选择可靠的伪标签困难;(ii)用伪标签微调LLM时可能引入标签噪声。为此提出GNN-as-Judge框架,通过引入图神经网络(GNN)的结构归纳偏置,释放LLM在少样本半监督学习中的潜力。具体地,该框架采用协同伪标签策略,先识别受已标注节点影响最大的未标注节点,再利用LLM与GNN之间的共识与分歧模式生成可靠标签。此外,设计一种弱监督微调算法,可从有信息量的伪标签中提炼知识,同时缓解标签噪声。在多个TAG数据集上的实验表明,GNN-as-Judge显著优于现有方法,尤其在标签稀缺的低资源条件下。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown strong performance on text-attributed graphs (TAGs) due to their superior semantic understanding ability on textual node features. However, their effectiveness as predictors in the low-resource setting, where labeled nodes are severely limited and scarce, remains constrained since fine-tuning LLMs usually requires sufficient labeled data, especially when the TAG shows complex structural patterns. In essence, this paper targets two key challenges: (i) the difficulty of generating and selecting reliable pseudo labels on TAGs for LLMs, and (ii) the need to mitigate potential label noise when fine-tuning LLMs with pseudo labels. To counter the challenges, we propose a new framework, GNN-as-Judge, which can unleash the power of LLMs for few-shot semi-supervised learning on TAGs by incorporating the structural inductive bias of Graph Neural Networks (GNNs). Specifically, GNN-as-Judge introduces a collaborative pseudo-labeling strategy that first identifies the most influenced unlabeled nodes from labeled nodes, then exploits both the agreement and disagreement patterns between LLMs and GNNs to generate reliable labels. Furthermore, we develop a weakly-supervised LLM fine-tuning algorithm that can distill the knowledge from informative pseudo labels while mitigating the potential label noise. Experiments on multiple TAG datasets demonstrate that GNN-as-Judge significantly outperforms existing methods, particularly in low-resource regimes where labeled data are scarce.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。