arXiv:2507.19031cs.LG2025-07AAAI被引 1

让GNN知识蒸馏到MLP时可按需调节速度与精度,适合不同算力环境。

ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs

  • 分阶段训练多个逐级增强的MLP学生模型,逐步提升性能。
  • 在8个真实图数据集上保持高准确率,支持动态调整推理开销。
  • 适合边缘计算、移动设备等资源受限场景部署使用。

GNN-to-MLP(G2M)方法通过将图神经网络的知识蒸馏到更简单的多层感知机(MLPs)中,成为加速GNN的有效途径。这类方法在保持较强表达能力的同时,显著提升了计算效率,适用于资源受限场景。然而,现有G2M方法难以灵活调整推理成本与准确率之间的权衡,难以适应实际应用中计算资源和时间约束的变化。为此,我们提出一种渐进式框架ProGMLP,实现GNN到MLP知识蒸馏中推理成本与精度的动态可调。ProGMLP采用渐进式训练结构(PTS),依次训练多个逐步增强的MLP学生模型;引入渐进式知识蒸馏(PKD),迭代优化从GNN到MLP的蒸馏过程;并结合渐进式混合增强(PMA),通过逐步生成更具挑战性的混合样本提升泛化能力。在八个真实世界图数据集上的实验表明,ProGMLP在不同运行时场景下均能保持高准确率,有效支持多样化部署需求。

原文摘要 · Abstract (English)

GNN-to-MLP (G2M) methods have emerged as a promising approach to accelerate Graph Neural Networks (GNNs) by distilling their knowledge into simpler Multi-Layer Perceptrons (MLPs). These methods bridge the gap between the expressive power of GNNs and the computational efficiency of MLPs, making them well-suited for resource-constrained environments. However, existing G2M methods are limited by their inability to flexibly adjust inference cost and accuracy dynamically, a critical requirement for real-world applications where computational resources and time constraints can vary significantly. To address this, we introduce a Progressive framework designed to offer flexible and on-demand trade-offs between inference cost and accuracy for GNN-to-MLP knowledge distillation (ProGMLP). ProGMLP employs a Progressive Training Structure (PTS), where multiple MLP students are trained in sequence, each building on the previous one. Furthermore, ProGMLP incorporates Progressive Knowledge Distillation (PKD) to iteratively refine the distillation process from GNNs to MLPs, and Progressive Mixup Augmentation (PMA) to enhance generalization by progressively generating harder mixed samples. Our approach is validated through comprehensive experiments on eight real-world graph datasets, demonstrating that ProGMLP maintains high accuracy while dynamically adapting to varying runtime scenarios, making it highly effective for deployment in diverse application settings.

知识蒸馏图神经网络模型压缩渐进式训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。