用10个参数快速学习高效数据采样器，提升模型训练效果
Swift Sampler: Efficient Learning of Sampler by 10 Parameters
- 将采样器映射为10个低维超参数，实现高效搜索
- 在ImageNet上使模型性能提升1.5%，且可跨网络迁移
- 计算成本低，适用于大规模数据集的自动化采样
数据选择对深度学习模型训练至关重要。有效的数据采样器能为训练数据分配合理的采样概率，帮助模型更快收敛至高性能的局部最优解。以往的数据采样方法多依赖启发式规则或需大量耗时试验进行学习。本文提出一种自动化的快速采样器搜索算法（SS），可高效探索有效采样器。具体而言，SS采用新公式将采样器映射至低维超参数空间，并利用近似局部最优快速评估采样器质量。得益于其低计算开销，SS可在大规模数据集上高效应用。在多种任务上的全面实验表明，由SS驱动的采样策略可带来显著提升（如ImageNet上提升1.5%），且在不同神经网络间具有良好的迁移能力。
原文摘要 · Abstract (English)
Data selection is essential for training deep learning models. An effective data sampler assigns proper sampling probability for training data and helps the model converge to a good local minimum with high performance. Previous studies in data sampling are mainly based on heuristic rules or learning through a huge amount of time-consuming trials. In this paper, we propose an automatic \textbf{swift sampler} search algorithm, \textbf{SS}, to explore automatically learning effective samplers efficiently. In particular, \textbf{SS} utilizes a novel formulation to map a sampler to a low dimension of hyper-parameters and uses an approximated local minimum to quickly examine the quality of a sampler. Benefiting from its low computational expense, \textbf{SS} can be applied on large-scale data sets with high efficiency. Comprehensive experiments on various tasks demonstrate that \textbf{SS} powered sampling can achieve obvious improvements (e.g., 1.5\% on ImageNet) and transfer among different neural networks. Project page: https://github.com/Alexander-Yao/Swift-Sampler.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。