用响应相关性评估网络架构，4小时在ImageNet上达到22.1%错误率
Zero-Shot Neural Architecture Search with Weighted Response Correlation
- 通过输入样本间响应相关性计算架构评分，无需训练
- 在ImageNet-1k上实现22.1%测试错误率，仅需4 GPU小时
- 适用于多种搜索空间，比现有方法更高效稳定
神经网络架构搜索（NAS）虽能自动设计模型，但传统方法需从头训练多个架构，计算开销巨大。现有零样本NAS方法虽避免训练，但评估效果、稳定性与泛化能力仍不足。本文提出一种新型无训练代理——加权响应相关性（WRCor），利用不同输入样本间响应的相关系数矩阵，衡量架构的表达能力与泛化性能。实验表明，WRCor及其投票策略在代理评估中优于现有方法。将其结合不同搜索策略应用于架构搜索，结果证明该零样本NAS算法在多个搜索空间均超越多数已有方法。在ImageNet-1k上，仅用4 GPU小时即发现测试错误率为22.1%的架构。代码已公开于https://github.com/kunjing96/ZSNAS-WRCor.git。
原文摘要 · Abstract (English)
Neural architecture search (NAS) is a promising approach for automatically designing neural network architectures. However, the architecture estimation of NAS is computationally expensive and time-consuming because of training multiple architectures from scratch. Although existing zero-shot NAS methods use training-free proxies to accelerate the architecture estimation, their effectiveness, stability, and generality are still lacking. We present a novel training-free estimation proxy called weighted response correlation (WRCor). WRCor utilizes correlation coefficient matrices of responses across different input samples to calculate the proxy scores of estimated architectures, which can measure their expressivity and generalizability. Experimental results on proxy evaluation demonstrate that WRCor and its voting proxies are more efficient estimation strategies than existing proxies. We also apply them with different search strategies in architecture search. Experimental results on architecture search show that our zero-shot NAS algorithm outperforms most existing NAS algorithms in different search spaces. Our NAS algorithm can discover an architecture with a 22.1% test error on the ImageNet-1k dataset within 4 GPU hours. All codes are publicly available at https://github.com/kunjing96/ZSNAS-WRCor.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。