Bonsai无需梯度,用计算树压缩图数据,训练快22倍且更准
Bonsai: Gradient-free Graph Condensation for Node Classification
- 以消息传递GNN的计算树为单位,选择代表性子树进行压缩
- 在7个真实数据集上准确率超基线,平均速度提升22倍
- 首次实现线性时间、模型无关的图压缩,适合快速迭代实验
图压缩已成为实现GNN可扩展训练的有前景方向,通过压缩训练数据集并保留关键图特征。但现有方法存在三大缺陷:多数算法需在全量数据上训练才能完成压缩;依赖梯度模拟,导致超参数或架构变化时需重新压缩;因生成全连接加权图,难以实现显著尺寸缩减。为此,我们提出Bonsai,其核心思想是:消息传递GNN的基本处理单元是计算树。Bonsai通过编码一组精心挑选的示例树来压缩数据集,最大化覆盖训练集中所有计算树的表示。该方法首次实现线性时间、模型无关的节点分类图压缩,在7个真实数据集上均超越现有基线,平均提速22倍,并具备严格的数学近似保证,对GNN架构、数据集和参数均鲁棒。
原文摘要 · Abstract (English)
Graph condensation has emerged as a promising avenue to enable scalable training of GNNs by compressing the training dataset while preserving essential graph characteristics. Our study uncovers significant shortcomings in current graph condensation techniques. First, the majority of the algorithms paradoxically require training on the full dataset to perform condensation. Second, due to their gradient-emulating approach, these methods require fresh condensation for any change in hyperparameters or GNN architecture, limiting their flexibility and reusability. Finally, they fail to achieve substantial size reduction due to synthesizing fully-connected, edge-weighted graphs. To address these challenges, we present Bonsai, a novel graph condensation method empowered by the observation that \textit{computation trees} form the fundamental processing units of message-passing GNNs. Bonsai condenses datasets by encoding a careful selection of \textit{exemplar} trees that maximize the representation of all computation trees in the training set. This unique approach imparts Bonsai as the first linear-time, model-agnostic graph condensation algorithm for node classification that outperforms existing baselines across $7$ real-world datasets on accuracy, while being $22$ times faster on average. Bonsai is grounded in rigorous mathematical guarantees on the adopted approximation strategies making it robust to GNN architectures, datasets, and parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。