将零样本NAS排名函数的波动视为随机变量，提升搜索性能
Variation Matters: from Mitigating to Embracing Zero-Shot NAS Ranking Function Variation
- 把排名函数输出看作随机变量，构建性能指标的随机排序
- 在标准基准上显著提升搜索效果，优于多次平均法
- 适合追求高精度NAS搜索的算法研究者
神经网络架构搜索（NAS）是自动设计神经网络的强大替代方法。在零样本版本中，通过快速排名函数在不训练的情况下比较网络架构。由于权重初始化或计算所用数据批次等随机因素，排名函数输出常存在显著差异。传统做法是多次评估后取平均以缓解波动。本文提出新思路：将排名函数输出视为代表性能的随机变量，在搜索过程中构建其随机排序，以确定最优架构。实验表明，该方法在标准基准搜索空间上能有效提升搜索性能。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) is a powerful automatic alternative to manual design of a neural network. In the zero-shot version, a fast ranking function is used to compare architectures without training them. The outputs of the ranking functions often vary significantly due to different sources of randomness, including the evaluated architecture's weights' initialization or the batch of data used for calculations. A common approach to addressing the variation is to average a ranking function output over several evaluations. We propose taking into account the variation in a different manner, by viewing the ranking function output as a random variable representing a proxy performance metric. During the search process, we strive to construct a stochastic ordering of the performance metrics to determine the best architecture. Our experiments show that the proposed stochastic ordering can effectively boost performance of a search on standard benchmark search spaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。