GDeR动态剪枝图数据,兼顾效率、平衡与鲁棒性
GDeR: Safeguarding Efficiency, Balancing, and Robustness via Prototypical Graph Pruning
- 基于可训练原型动态更新训练样本池,从嵌入空间采样代表性子集
- 用30%~50%数据达到全量数据性能,训练提速2.81倍且无损失
- 适合处理不平衡/含噪图数据,对边缘设备部署有重要意义
训练高质量深度模型需要大量数据,带来巨大的计算与内存开销。数据剪枝、蒸馏和核心集选择等方法通过保留、合成或选取少量信息丰富子集来缩减数据量,其中剪枝成本最低、加速效果最显著。然而,它在数据不平衡或有偏时易性能下降,影响其在设备端部署的可靠性。现有方案在图像与自然语言领域已较成熟,但图神经网络(GNN)仍面临大规模、不平衡、噪声数据挑战,缺乏统一的数据剪枝方案。为此,我们提出新型动态软剪枝方法GDeR,通过可训练原型在训练中更新“训练篮子”。GDeR首先构建建模良好的图嵌入超球面,从中采样出代表性强、平衡且无偏的子集,实现所谓“图训练调试”。在三个GNN主干和五个数据集上的实验表明:GDeR(I)仅用30%~50%训练样本即达全量数据性能;(II)实现最高2.81倍无损训练加速;(III)在不平衡与噪声训练场景下,相比最先进剪枝方法提升0.3%~4.3%和3.6%~7.8%。
原文摘要 · Abstract (English)
Training high-quality deep models necessitates vast amounts of data, resulting in overwhelming computational and memory demands. Recently, data pruning, distillation, and coreset selection have been developed to streamline data volume by retaining, synthesizing, or selecting a small yet informative subset from the full set. Among these methods, data pruning incurs the least additional training cost and offers the most practical acceleration benefits. However, it is the most vulnerable, often suffering significant performance degradation with imbalanced or biased data schema, thus raising concerns about its accuracy and reliability in on-device deployment. Therefore, there is a looming need for a new data pruning paradigm that maintains the efficiency of previous practices while ensuring balance and robustness. Unlike the fields of computer vision and natural language processing, where mature solutions have been developed to address these issues, graph neural networks (GNNs) continue to struggle with increasingly large-scale, imbalanced, and noisy datasets, lacking a unified dataset pruning solution. To achieve this, we introduce a novel dynamic soft-pruning method, GDeR, designed to update the training ``basket'' during the process using trainable prototypes. GDeR first constructs a well-modeled graph embedding hypersphere and then samples \textit{representative, balanced, and unbiased subsets} from this embedding space, which achieves the goal we called Graph Training Debugging. Extensive experiments on five datasets across three GNN backbones, demonstrate that GDeR (I) achieves or surpasses the performance of the full dataset with 30%~50% fewer training samples, (II) attains up to a 2.81x lossless training speedup, and (III) outperforms state-of-the-art pruning methods in imbalanced training and noisy training scenarios by 0.3%~4.3% and 3.6%~7.8%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。