arXiv:2411.06236cs.LGcs.CV2024-11被引 2

提出无需训练和数据的零样本搜索新代理,加速架构评估

Zero-Shot NAS via the Suppression of Local Entropy Decrease

  • 用网络拓扑衡量局部熵下降抑制程度作为代理
  • 在五个基准上超越多数先进方法,计算提速千倍
  • 适合快速筛选高性能轻量级模型

神经架构搜索(NAS)中架构性能评估最为耗时。零样本NAS通过零成本代理替代训练来加速评估。现有代理需反向传播或运行网络于输入数据,难以进一步加速。本文利用网络拓扑评估性能,证明特定拓扑会降低特征图的局部熵,导致特征偏置,损害网络性能。基于此,提出一种数据无关、无需运行的代理——局部熵下降抑制(SED)。实验表明,SED在五个基准上优于多数先进代理,计算时间减少三个数量级。基于SED的NAS仅用1秒即可选出准确率更高、参数更少的架构。理论分析与实验证明,抑制局部熵下降有助于在零样本NAS中选择最优架构。

原文摘要 · Abstract (English)

Architecture performance evaluation is the most time-consuming part of neural architecture search (NAS). Zero-Shot NAS accelerates the evaluation by utilizing zero-cost proxies instead of training. Though effective, existing zero-cost proxies require invoking backpropagations or running networks on input data, making it difficult to further accelerate the computation of proxies. To alleviate this issue, architecture topologies are used to evaluate the performance of networks in this study. We prove that particular architectural topologies decrease the local entropy of feature maps, which degrades specific features to a bias, thereby reducing network performance. Based on this proof, architectural topologies are utilized to quantify the suppression of local entropy decrease (SED) as a data-free and running-free proxy. Experimental results show that SED outperforms most state-of-the-art proxies in terms of architecture selection on five benchmarks, with computation time reduced by three orders of magnitude. We further compare the SED-based NAS with state-of-the-art proxies. SED-based NAS selects the architecture with higher accuracy and fewer parameters in only one second. The theoretical analyses of local entropy and experimental results demonstrate that the suppression of local entropy decrease facilitates selecting optimal architectures in Zero-Shot NAS.

零样本NAS架构搜索代理模型效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。