RCAP动态裁剪数据,提升不平衡数据下最差类别准确率。
RCAP: Robust, Class-Aware, Probabilistic Dynamic Dataset Pruning

- 按类别自适应计算每轮保留样本比例,基于损失加权采样。
- 仅用10%数据仍比全量训练高1%以上准确率,平均提速8.69倍。
- 适合追求高效训练且关注少数类性能的分类任务研究者。
动态数据裁剪通过在训练过程中周期性选择代表性数据子集,以降低计算成本并最小化信息损失。然而,现有方法在高裁剪率下难以保持强健的最差组准确率,尤其在平衡与不平衡数据集上表现不佳。为此,我们提出RCAP——一种鲁棒、类别感知、概率化的动态数据裁剪算法,适用于分类任务。RCAP采用闭式解估算每个类别在训练子集中应包含的样本比例,并在每轮中根据类别聚合损失自适应调整该比例。随后,采用自适应采样策略,优先保留高损失样本以填充各类别子集。我们在六个不同数据集(从平衡到高度不平衡)上,使用五种模型及三种训练范式(从头训练、迁移学习、微调)评估了RCAP。结果表明,该方法在所有裁剪率下均显著优于现有最优方法,持续提升最差组准确率。特别地,仅使用10%数据时,RCAP在类别不平衡数据集上性能优于全量数据训练超过1%,同时实现平均8.69倍加速。代码已开源于https://github.com/atif-hassan/RCAP-dynamic-dataset-pruning。
原文摘要 · Abstract (English)
Dynamic data pruning techniques aim to reduce computational cost while minimizing information loss by periodically selecting representative subsets of input data during model training. However, existing methods often struggle to maintain strong worst-group accuracy, particularly at high pruning rates, across balanced and imbalanced datasets. To address this challenge, we propose RCAP, a Robust, Class-Aware, Probabilistic dynamic dataset pruning algorithm for classification tasks. RCAP applies a closed-form solution to estimate the fraction of samples to be included in the training subset for each individual class. This fraction is adaptively adjusted in every epoch using class-wise aggregated loss. Thereafter, it employs an adaptive sampling strategy that prioritizes samples having high loss for populating the class-wise subsets. We evaluate RCAP on six diverse datasets ranging from class-balanced to highly imbalanced using five distinct models across three training paradigms: training from scratch, transfer learning, and fine-tuning. Our approach consistently outperforms state-of-the-art dataset pruning methods, achieving superior worst-group accuracy at all pruning rates. Remarkably, with only $10\%$ data, RCAP delivers $>1\%$ improvement in performance on class-imbalanced datasets compared to full data training while providing an average $8.69\times$ speedup. The code can be accessed at https://github.com/atif-hassan/RCAP-dynamic-dataset-pruning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。