arXiv:2510.19223cs.LG2025-10

GNN模型互学互促,无需教师模型也能提升性能。

Enhancing Graph Neural Networks: A Mutual Learning Approach

  • 多GNN学生模型互相教学,动态调整学习策略。
  • 在三个数据集上实现优于单模型的节点与图分类效果。
  • 适合资源受限场景下部署高效多任务GNN系统。

知识蒸馏(KD)技术已成为将复杂教师模型的知识迁移至轻量级学生模型的强大工具,尤其适用于在资源受限设备上部署高性能模型。该方法已被成功应用于图神经网络(GNN),利用其表达能力生成捕捉结构与特征信息的节点嵌入。本文突破传统KD范式,探索了GNN之间在无预训练教师模型情况下协同学习的潜力。我们发现,相对简单浅层的GNN架构可通过协同学习,联合优化出推理表现更优的高效模型,尤其在多任务场景中优势显著。为此,我们提出一种协同学习框架:由多个学生GNN模型在训练过程中相互教学。引入自适应logit加权单元以促进模型间高效知识交换,并采用熵增强技术提升互学效果。这些组件使模型能在训练中动态调整学习策略,从而优化下游任务表现。在三个分别用于节点分类和图分类的数据集上开展的大量实验验证了本方法的有效性。

原文摘要 · Abstract (English)

Knowledge distillation (KD) techniques have emerged as a powerful tool for transferring expertise from complex teacher models to lightweight student models, particularly beneficial for deploying high-performance models in resource-constrained devices. This approach has been successfully applied to graph neural networks (GNNs), harnessing their expressive capabilities to generate node embeddings that capture structural and feature-related information. In this study, we depart from the conventional KD approach by exploring the potential of collaborative learning among GNNs. In the absence of a pre-trained teacher model, we show that relatively simple and shallow GNN architectures can synergetically learn efficient models capable of performing better during inference, particularly in tackling multiple tasks. We propose a collaborative learning framework where ensembles of student GNNs mutually teach each other throughout the training process. We introduce an adaptive logit weighting unit to facilitate efficient knowledge exchange among models and an entropy enhancement technique to improve mutual learning. These components dynamically empower the models to adapt their learning strategies during training, optimizing their performance for downstream tasks. Extensive experiments conducted on three datasets each for node and graph classification demonstrate the effectiveness of our approach.

图神经网络知识蒸馏协同学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。