用聚类方法高效压缩图数据,训练更快性能不降。
Simple yet Effective Graph Distillation via Clustering
- 基于最小化簇内平方和与最大化同质性设计聚类策略
- 在5个基准数据集上性能优于或媲美现有方法,速度提升数个数量级
- 适合需要快速训练图神经网络的研究者和工程实践
尽管图表示学习在多个领域取得成功,但图神经网络(GNN)在实际中因大规模图带来的巨大计算开销而难以训练。近年来,图数据蒸馏(GDD)通过将大图浓缩为紧凑且信息丰富的小图,成为实现高效GNN训练的有前景技术。然而,多数现有GDD方法依赖对齐模型梯度或表示分布的启发式策略,导致结果质量下降、蒸馏过程昂贵。本文提出一种高效有效的GDD方法ClustGDD:通过快速且理论严谨的聚类,最小化簇内平方和并最大化原始图上的同质性来合成压缩图及其节点属性。该思路源于我们通过弗雷歇起始距离(FID)发现的聚类与蒸馏质量间的联系。此外,为缓解同质性聚类的负面影响,ClustGDD利用类别感知图采样与一致性损失学习微小增强,优化压缩图节点属性。大量实验表明,在五个基准数据集上,基于ClustGDD生成的压缩图训练的GNN,其节点分类性能始终优于或媲美当前最优方法,同时速度提升数个数量级。
原文摘要 · Abstract (English)
Despite plentiful successes achieved by graph representation learning in various domains, the training of graph neural networks (GNNs) still remains tenaciously challenging due to the tremendous computational overhead needed for sizable graphs in practice. Recently, graph data distillation (GDD), which seeks to distill large graphs into compact and informative ones, has emerged as a promising technique to enable efficient GNN training. However, most existing GDD works rely on heuristics that align model gradients or representation distributions on condensed and original graphs, leading to compromised result quality, expensive training for distilling large graphs, or both. Motivated by this, this paper presents an efficient and effective GDD approach, ClustGDD. Under the hood, ClustGDD resorts to synthesizing the condensed graph and node attributes through fast and theoretically-grounded clustering that minimizes the within-cluster sum of squares and maximizes the homophily on the original graph. The fundamental idea is inspired by our empirical and theoretical findings unveiling the connection between clustering and empirical condensation quality using Fréchet Inception Distance, a well-known quality metric for synthetic images. Furthermore, to mitigate the adverse effects caused by the homophily-based clustering, ClustGDD refines the nodal attributes of the condensed graph with a small augmentation learned via class-aware graph sampling and consistency loss. Our extensive experiments exhibit that GNNs trained over condensed graphs output by ClustGDD consistently achieve superior or comparable performance to state-of-the-art GDD methods in terms of node classification on five benchmark datasets, while being orders of magnitude faster.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。