自动设计零成本代理,高效评估神经网络以降低能耗。
GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAS
- 用进化算法自动生成零成本性能代理。
- 在CIFAR-10/100上达0.89/0.77的肯德尔相关性。
- 适合关注绿色AI与高效NAS的研究者。
人工智能推动了各领域的创新,但深度神经网络的模型设计依赖神经架构搜索(NAS),该过程资源与时间消耗大,带来高能耗问题,形成效率与可持续性的矛盾。为此,零成本代理成为替代方案,可在不训练的情况下预测模型性能。本文提出自动设计零成本代理的方法:从随机生成的代理集出发,通过NATS-Bench基准进行演化与测试,使用随机与分层采样子集评估其区分低/高性能网络的能力和泛化性。结果表明,该方法在分层采样策略下表现更优,在CIFAR-10上的肯德尔相关性达0.89,CIFAR-100为0.77;在全空间采样策略下,对应相关性分别为0.78和0.71。
原文摘要 · Abstract (English)
Artificial Intelligence (AI) has driven innovations and created new opportunities across various sectors. However, leveraging domain-specific knowledge often requires automated tools to design and configure models effectively. In the case of Deep Neural Networks (DNNs), researchers and practitioners usually resort to Neural Architecture Search (NAS) approaches, which are resource- and time-intensive, requiring the training and evaluation of numerous candidate architectures. This raises sustainability concerns, particularly due to the high energy demands involved, creating a paradox: the pursuit of the most effective model can undermine sustainability goals. To mitigate this issue, zero-cost proxies have emerged as a promising alternative. These proxies estimate a model's performance without the need for full training, offering a more efficient approach. This paper addresses the challenges of model evaluation by automatically designing zero-cost proxies to assess DNNs efficiently. Our method begins with a randomly generated set of zero-cost proxies, which are evolved and tested using the NATS-Bench benchmark. We assess the proxies' effectiveness using both randomly sampled and stratified subsets of the search space, ensuring they can differentiate between low- and high-performing networks and enhance generalizability. Results show our method outperforms existing approaches on the stratified sampling strategy, achieving strong correlations with ground truth performance, including a Kendall correlation of 0.89 on CIFAR-10 and 0.77 on CIFAR-100 with NATS-Bench-SSS and a Kendall correlation of 0.78 on CIFAR-10 and 0.71 on CIFAR-100 with NATS-Bench-TSS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。