不训练网络就能高效评估架构性能，大幅提升搜索效率。
Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weights
- 基于费舍尔信息设计无训练代理指标，理论依据强
- 在三个数据集上达最优，计算成本降低超90%
- 适合快速筛选架构，尤其适用于资源受限场景
深度学习在计算机视觉领域取得巨大成功，但其核心网络架构多为人工设计，可能存在次优性。神经架构搜索（NAS）旨在通过系统优化寻找最佳架构，但主要瓶颈在于每次候选架构需从头训练，计算成本极高。本文提出一种全新的无训练代理指标，基于费舍尔信息构建，可在不训练网络的情况下估算图像分类准确率，显著降低标准NAS算法的计算开销。该方法在三个公开数据集和两个搜索空间中均达到当前最佳表现，且在新提出的更实用的评估指标下也表现出更强的信息量。代码已开源。
原文摘要 · Abstract (English)
Deep learning has revolutionized computer vision, but it achieved its tremendous success using deep network architectures which are mostly hand-crafted and therefore likely suboptimal. Neural Architecture Search (NAS) aims to bridge this gap by following a well-defined optimization paradigm which systematically looks for the best architecture, given objective criterion such as maximal classification accuracy. The main limitation of NAS is however its astronomical computational cost, as it typically requires training each candidate network architecture from scratch. In this paper, we aim to alleviate this limitation by proposing a novel training-free proxy for image classification accuracy based on Fisher Information. The proposed proxy has a strong theoretical background in statistics and it allows estimating expected image classification accuracy of a given deep network without training the network, thus significantly reducing computational cost of standard NAS algorithms. Our training-free proxy achieves state-of-the-art results on three public datasets and in two search spaces, both when evaluated using previously proposed metrics, as well as using a new metric that we propose which we demonstrate is more informative for practical NAS applications. The source code is publicly available at http://www.github.com/ondratybl/VKDNW
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。