用层次聚类优化搜索顺序，让神经网络架构搜索更高效
Neural Architecture Search by Learning a Hierarchical Search Space
- 通过聚类相似性构建分层搜索结构，引导搜索优先探索优质路径
- 在CIFAR10和ImageNet上比传统方法更快找到性能优异的模型
- 适合需要高效架构搜索的研究者或工业部署场景
蒙特卡洛树搜索（MCTS）在非可微搜索问题中表现优异，但其效率高度依赖于树分支时节点的访问顺序。若初始分支无法区分有前途与误导性的架构配置，搜索效率将指数级下降。在神经网络架构搜索（NAS）中，只有最终架构有效，因此可优化分支顺序以提升学习效率。本文研究MCTS在图像分类任务中的应用，分析多种采样策略与分支方式，提出基于架构输出向量相似性进行层次聚类，以学习最优分支顺序。在两个挑战性基准——CIFAR10和ImageNet上的大量实验表明，若提供良好的分支层次结构，MCTS能比其他方法更高效地发现性能优异的模型。
原文摘要 · Abstract (English)
Monte-Carlo Tree Search (MCTS) is a powerful tool for many non-differentiable search related problems such as adversarial games. However, the performance of such approach highly depends on the order of the nodes that are considered at each branching of the tree. If the first branches cannot distinguish between promising and deceiving configurations for the final task, the efficiency of the search is exponentially reduced. In Neural Architecture Search (NAS), as only the final architecture matters, the visiting order of the branching can be optimized to improve learning. In this paper, we study the application of MCTS to NAS for image classification. We analyze several sampling methods and branching alternatives for MCTS and propose to learn the branching by hierarchical clustering of architectures based on their similarity. The similarity is measured by the pairwise distance of output vectors of architectures. Extensive experiments on two challenging benchmarks on CIFAR10 and ImageNet show that MCTS, if provided with a good branching hierarchy, can yield promising solutions more efficiently than other approaches for NAS problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。