用元学习优化训练率，降低搜索成本并提升模型鲁棒性。
Meta knowledge assisted Evolutionary Neural Architecture Search
- 通过元学习预训练获取自适应学习率策略，减少评估误差。
- 设计自适应代理模型，在少轮内筛选优质架构，节省计算资源。
- 引入周期性变异增强种群多样性，适合高效神经网络搜索场景。
基于进化计算的神经架构搜索（NAS）在自动设计神经网络方面表现优异，但高计算开销和固定学习率调度导致信息损失。本文提出一种高效进化式NAS方法，通过创新的元学习框架解决上述问题。首先，采用元学习率（Meta-LR）策略，通过预训练获得适配各架构的动态学习率，降低评估过程中的信息损失；其次，设计自适应代理模型，基于动态阈值在少数训练轮次内筛选有潜力的候选架构，再以完整轮次进行精确评估；此外，提出周期性变异算子，提升种群多样性，增强算法泛化能力与鲁棒性。在CIFAR-10、CIFAR-100和ImageNet1K上的实验表明，该方法性能接近多项先进方法，同时计算成本更低、鲁棒性更强。
原文摘要 · Abstract (English)
Evolutionary computation (EC)-based neural architecture search (NAS) has achieved remarkable performance in the automatic design of neural architectures. However, the high computational cost associated with evaluating searched architectures poses a challenge for these methods, and a fixed form of learning rate (LR) schedule means greater information loss on diverse searched architectures. This paper introduces an efficient EC-based NAS method to solve these problems via an innovative meta-learning framework. Specifically, a meta-learning-rate (Meta-LR) scheme is used through pretraining to obtain a suitable LR schedule, which guides the training process with lower information loss when evaluating each individual. An adaptive surrogate model is designed through an adaptive threshold to select the potential architectures in a few epochs and then evaluate the potential architectures with complete epochs. Additionally, a periodic mutation operator is proposed to increase the diversity of the population, which enhances the generalizability and robustness. Experiments on CIFAR-10, CIFAR-100, and ImageNet1K datasets demonstrate that the proposed method achieves high performance comparable to that of many state-of-the-art peer methods, with lower computational cost and greater robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。