用图模型统一处理数据集剪枝,提速40%不丢精度
Selecting Samples on Graphs: A Unified Dataset Pruning Framework for Lossless Training Acceleration
- 将样本建模为带权图,节点权表内在价值,边权表外部关系
- 通过贪心算法求解最大权重团,实现40%以上训练加速
- 理论保证适用于多种评估指标,适合大规模图像训练
现代训练数据集规模激增导致计算成本上升,促使数据集剪枝(DP)方法的发展,旨在保留信息量高的样本以降低训练开销。现有剪枝标准通常依赖于独立评估样本的内在信号,或通过成对关系促进多样性的外在信号。二者虽在特定场景有效,但仅捕捉样本效用的一个方面,且在不同剪枝比例或数据分布下缺乏鲁棒性。本文提出一种统一的图基剪枝框架:将数据集建模为加权图,节点权重表示内在价值,边权重表示外在价值,剪枝问题转化为最大权重团问题(MWCP)。尽管该问题是NP难,但其结构支持基于样本边际增益的合理贪心解法。在若干温和条件下,我们进一步证明该统一目标具有形式化近似保证,适用于广泛的重要性度量,并提供实用的设计指导。大量实验表明,本方法优于现有剪枝方法,在ImageNet-1k上使用ResNet-50时,训练时间减少超40%且准确率不变。
原文摘要 · Abstract (English)
The rapid growth of modern training datasets has significantly increased computational cost, motivating dataset pruning~(DP) methods which retain only a subset of informative samples to reduce training cost. Existing pruning criteria typically rely on either intrinsic signals that assess samples independently or extrinsic signals that promote diversity via pairwise relations. While effective in their own specific regimes, each captures only one aspect of sample utility and lacks robustness across different pruning ratios or data distribution. In this work, we present a unified graph-based DP framework. By modeling the dataset as a weighted graph, where node weights encode intrinsic value and edge weights encode extrinsic value, DP can be cast as a Maximum Weight Clique Problem (MWCP). Although MWCP is NP-hard, its structure admits a principled greedy solution based on sample-wise marginal gains. Under a few mild conditions, we further prove that this unified objective enjoys a formal approximation guarantee, which applies to a broad family of importance metrics and provides practical design guidelines. Extensive experiments show that our method outperforms existing DP methods while substantially reducing training cost, reducing training time by over 40\% without sacrificing accuracy on ImageNet-1k with ResNet-50.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。