用遗传编程自动设计零成本性能预测器,比人工设计更准更快。
From Hand-Crafted Metrics to Evolved Training-Free Performance Predictors for Neural Architecture Search via Genetic Programming
- 通过遗传编程自动合成零成本性能预测指标。
- 在13个NAS任务中,新指标与真实性能相关性显著提升。
- 仅用单张消费级显卡,15分钟内找到高性能模型。
使用零成本(ZC)指标估算网络性能已在神经架构搜索(NAS)中证明其高效与有效。然而,大多数ZC代理存在不一致性,表现为在不同问题间性能波动大。此外,现有ZC指标的设计为人工构造,需耗时试错且依赖大量领域知识。这引出两个关键问题:(1) 能否自动化设计ZC指标?(2) 能否利用已有手工设计的ZC指标合成更具泛化能力的新指标?本研究提出一种基于符号回归的遗传编程框架,实现ZC指标的自动化设计。该框架高度可扩展,能快速生成在多种NAS搜索空间和任务上均表现出强正秩相关性的ZC指标。在13个来自NAS-Bench-Suite-Zero的问题上进行的大量实验表明,所生成的代理指标始终优于人工设计的替代方案。将该进化后的代理指标用于进化算法的搜索目标,仅用单张消费级GPU,在15分钟内即可找到性能有竞争力的网络架构。
原文摘要 · Abstract (English)
Estimating the network performance using zero-cost (ZC) metrics has proven both its efficiency and efficacy in Neural Architecture Search (NAS). However, a notable limitation of most ZC proxies is their inconsistency, as reflected by the substantial variation in their performance across different problems. Furthermore, the design of existing ZC metrics is manual, involving a time-consuming trial-and-error process that requires substantial domain expertise. These challenges raise two critical questions: (1) Can we automate the design of ZC metrics? and (2) Can we utilize the existing hand-crafted ZC metrics to synthesize a more generalizable one? In this study, we propose a framework based on Symbolic Regression via Genetic Programming to automate the design of ZC metrics. Our framework is not only highly extensible but also capable of quickly producing a ZC metric with a strong positive rank correlation to true network performance across diverse NAS search spaces and tasks. Extensive experiments on 13 problems from NAS-Bench-Suite-Zero demonstrate that our automatically generated proxies consistently outperform hand-crafted alternatives. Using our evolved proxy metric as the search objective in an evolutionary algorithm, we could identify network architectures with competitive performance within 15 minutes using a single consumer GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。