用谱分析筛选关键数据,训练效率提升4倍且不降精度。
Efficient Training of Deep Networks using Guided Spectral Data Selection: A Step Toward Learning What You Need
- 基于预训练模型和谱分析动态选数据,每批减少计算量。
- 在CIFAR-10等数据集上降低计算量达4倍,准确率反升。
- 适合资源受限场景,推动自适应数据选择研究。
有效的数据整理对优化神经网络训练至关重要。本文提出引导谱调谐数据选择(GSTDS)算法,利用现成的预训练参考模型动态调整每批次训练的数据子集。根据预设的过滤比例,GSTDS显著减少每批处理的数据点数。该方法通过谱分析保留每批中最具有信息量的数据点,避免冗余计算。采用费德勒向量评分机制剔除低信息量部分,减轻学习资源负担。实验表明,该方法不仅简化了训练流程,还提升了泛化能力与准确率。在标准图像分类基准(如CIFAR-10、Oxford-IIIT Pet、Oxford-Flowers)上,相比标准训练和近期先进方法JEST,GSTDS在多个关键指标上表现更优,计算量最高降低四倍,且性能未下降。结果证明谱基数据选择是资源高效深度学习的可扩展方案,激励进一步探索自适应数据整理策略。代码见:https://github.com/rezasharifi82/GSTDS。
原文摘要 · Abstract (English)
Effective data curation is essential for optimizing neural network training. In this paper, we present the Guided Spectrally Tuned Data Selection (GSTDS) algorithm, which dynamically adjusts the subset of data points used for training using an off-the-shelf pre-trained reference model. Based on a pre-scheduled filtering ratio, GSTDS effectively reduces the number of data points processed per batch. The proposed method ensures an efficient selection of the most informative data points for training while avoiding redundant or less beneficial computations. Preserving data points in each batch is performed based on spectral analysis. A Fiedler vector-based scoring mechanism removes the filtered portion of the batch, lightening the resource requirements of the learning. The proposed data selection approach not only streamlines the training process but also promotes improved generalization and accuracy. Extensive experiments on standard image classification benchmarks, including CIFAR-10, Oxford-IIIT Pet, and Oxford-Flowers, demonstrate that GSTDS outperforms standard training scenarios and JEST, a recent state-of-the-art data curation method, on several key factors. It is shown that GSTDS achieves notable reductions in computational requirements, up to four times, without compromising performance. GSTDS exhibits a considerable growth in terms of accuracy under the limited computational resource usage, in contrast to other methodologies. These promising results underscore the potential of spectral-based data selection as a scalable solution for resource-efficient deep learning and motivate further exploration into adaptive data curation strategies. You can find the code at https://github.com/rezasharifi82/GSTDS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。