用用户偏好提升小样本节点分类效果
Preference-driven Knowledge Distillation for Few-shot Node Classification
- 根据节点特征选择最合适的教师GNN进行知识蒸馏
- 结合大模型与GNN，实现少样本场景下精准分类
- 适合处理复杂图结构中的小样本学习任务
图神经网络（GNN）能高效处理文本属性图（TAGs），但其训练高度依赖人工标注标签。真实世界的TAGs节点局部拓扑复杂多样，单一机制难以应对。大语言模型（LLMs）在TAGs上表现良好，但在少样本/零样本学习中存在可扩展性挑战。为此，我们提出一种偏好驱动的知识蒸馏（PKD）框架，融合LLMs与多种GNN的互补优势，用于少样本节点分类。具体地，设计基于GNN偏好的节点选择器，促进从LLMs向教师GNN的知识蒸馏；进一步设计基于节点偏好的GNN选择器，为每个节点识别最适配的教师GNN，实现定制化知识迁移。大量实验验证了该框架在真实世界TAGs上的有效性。代码将公开。
原文摘要 · Abstract (English)
Graph neural networks (GNNs) can efficiently process text-attributed graphs (TAGs) due to their message-passing mechanisms, but their training heavily relies on the human-annotated labels. Moreover, the complex and diverse local topologies of nodes of real-world TAGs make it challenging for a single mechanism to handle. Large language models (LLMs) perform well in zero-/few-shot learning on TAGs but suffer from a scalability challenge. Therefore, we propose a preference-driven knowledge distillation (PKD) framework to synergize the complementary strengths of LLMs and various GNNs for few-shot node classification. Specifically, we develop a GNN-preference-driven node selector that effectively promotes prediction distillation from LLMs to teacher GNNs. To further tackle nodes' intricate local topologies, we develop a node-preference-driven GNN selector that identifies the most suitable teacher GNN for each node, thereby facilitating tailored knowledge distillation from teacher GNNs to the student GNN. Extensive experiments validate the efficacy of our proposed framework in few-shot node classification on real-world TAGs. Our code is be available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。