通过动态选难例加速神经架构搜索，训练速度提升27倍且性能不降。
DDS-NAS: Dynamic Data Selection within Neural Architecture Search via On-line Hard Example Mining applied to Image Classification
- 基于自编码器构建图像相似性嵌入，用kd树快速定位最不相似样本。
- 在课程学习框架下动态筛选模型表现差的难例，形成高效子数据集。
- 适合追求高速神经架构搜索的开发者，尤其适用于大规模图像分类任务。
为解决神经架构搜索（NAS）的可扩展性挑战，本文提出一种基于在线难例挖掘的动态数据选择方法（DDS-NAS），在课程学习框架内加速NAS训练。通过自编码器在低维嵌入空间中强制图像相似性表示，构建高效的kd树结构，以对数时间复杂度从全局数据集中定位与查询图像最不相似的样本。基于此，课程学习机制动态重构当前架构表现不佳的无偏子数据集，用于NAS优化。实验表明，该框架使基于梯度的NAS策略训练速度最高提升27倍，且性能无损失。通过最大化每个样本的贡献，显著缩短单轮NAS训练时长，并减少收敛所需迭代次数。
原文摘要 · Abstract (English)
In order to address the scalability challenge within Neural Architecture Search (NAS), we speed up NAS training via dynamic hard example mining within a curriculum learning framework. By utilizing an autoencoder that enforces an image similarity embedding in latent space, we construct an efficient kd-tree structure to order images by furthest neighbour dissimilarity in a low-dimensional embedding. From a given query image from our subsample dataset, we can identify the most dissimilar image within the global dataset in logarithmic time. Via curriculum learning, we then dynamically re-formulate an unbiased subsample dataset for NAS optimisation, upon which the current NAS solution architecture performs poorly. We show that our DDS-NAS framework speeds up gradient-based NAS strategies by up to 27x without loss in performance. By maximising the contribution of each image sample during training, we reduce the duration of a NAS training cycle and the number of iterations required for convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。