提出高效全局神经网络架构搜索框架,兼顾精度与速度。
Efficient Global Neural Architecture Search
- 构建可导航的宏微协同搜索空间,支持端到端自动化设计。
- 采用自适应训练策略,提升不同网络间的公平性评估。
- 在多个数据集上超越现有方法,速度提升2-4倍,适合实际部署。
神经网络架构搜索(NAS)虽有望自动化网络设计,但因需大量训练评估而计算成本高昂。现有方法多采用模块化搜索,忽略网络宏观结构(深度与宽度),依赖人工调参,缺乏完全自动化。本文重新审视全局搜索,设计可导航且多样化的宏微协同搜索空间。针对不同网络采用差异化的训练方案进行性能近似,避免统一协议带来的不公平比较。通过分离宏观与微观设计的搜索策略,实现高效率的架构探索,在EMNIST和KMNIST上达到新SOTA,CIFAR-10、CIFAR-100和FashionMNIST上表现优异,比最快全局搜索方法快2-4倍。此外,验证了该框架在人脸识别等真实视觉任务中的可迁移性。
原文摘要 · Abstract (English)
Neural architecture search (NAS) has shown promise towards automating neural network design for a given task, but it is computationally demanding due to training costs associated with evaluating a large number of architectures to find the optimal one. To speed up NAS, recent works limit the search to network building blocks (modular search) instead of searching the entire architecture (global search), approximate candidates' performance evaluation in lieu of complete training, and use gradient descent rather than naturally suitable discrete optimization approaches. However, modular search does not determine network's macro architecture i.e. depth and width, demanding manual trial and error post-search, hence lacking automation. In this work, we revisit NAS and design a navigable, yet architecturally diverse, macro-micro search space. In addition, to determine relative rankings of candidates, existing methods employ consistent approximations across entire search spaces, whereas different networks may not be fairly comparable under one training protocol. Hence, we propose an architecture-aware approximation with variable training schemes for different networks. Moreover, we develop an efficient search strategy by disjoining macro-micro network design that yields competitive architectures in terms of both accuracy and size. Our proposed framework achieves a new state-of-the-art on EMNIST and KMNIST, while being highly competitive on the CIFAR-10, CIFAR-100, and FashionMNIST datasets and being 2-4x faster than the fastest global search methods. Lastly, we demonstrate the transferability of our framework to real-world computer vision problems by discovering competitive architectures for face recognition applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。