通过频谱分析在批次内选关键样本，大幅提速并提效训练。
Optimizing Data Curation through Spectral Analysis and Joint Batch Selection (SALN)
- 在每个批次内用频谱分析筛选最有信息量的样本
- 训练时间最多减少8倍，准确率提升最高5%
- 适合追求高效训练的深度学习研究者和工程师
现代深度学习模型面临训练时间长、数据量大的挑战。有效的数据清洗与样本选择对优化神经网络训练至关重要。本文提出SALN方法，不从全数据集选择样本，而是优先处理每个批次内的样本。通过联合选取批次，SALN相比独立批次选择提升了训练效率。该方法采用基于频谱分析的启发式策略，识别每批次中最具信息量的数据点，从而加速训练并提高准确率。与传统批次优先或标准训练相比，SALN显著缩短训练时间，提升准确率，实现最多8倍的时间缩减和最高5%的精度提升。此外，其性能优于DeepMind开发的Google JEST方法。
原文摘要 · Abstract (English)
In modern deep learning models, long training times and large datasets present significant challenges to both efficiency and scalability. Effective data curation and sample selection are crucial for optimizing the training process of deep neural networks. This paper introduces SALN, a method designed to prioritize and select samples within each batch rather than from the entire dataset. By utilizing jointly selected batches, SALN enhances training efficiency compared to independent batch selection. The proposed method applies a spectral analysis-based heuristic to identify the most informative data points within each batch, improving both training speed and accuracy. The SALN algorithm significantly reduces training time and enhances accuracy when compared to traditional batch prioritization or standard training procedures. It demonstrates up to an 8x reduction in training time and up to a 5\% increase in accuracy over standard training methods. Moreover, SALN achieves better performance and shorter training times compared to Google's JEST method developed by DeepMind.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。