用可迁移代理模型加速复杂架构搜索，提升效率与性能。
Transferrable Surrogates in Expressive Neural Architecture Search Spaces
- 基于语法生成的表达性搜索空间，训练可迁移的代理模型
- 跨数据集预测准确，过滤劣质架构使搜索提速并优化结果
- 代理模型可直接作为搜索目标，实现极高速度提升
神经架构搜索（NAS）面临如何在表达性强、范围广的搜索空间中探索创新架构，同时保持高效评估的挑战。本文研究基于无上下文语法的高表达性NAS搜索空间中的代理模型训练方法。实验表明：i）使用零成本代理指标和神经图特征（GRAF），或微调现成语言模型（LM）训练的代理模型，在跨数据集场景下均具备强性能预测能力；ii）这些代理模型可用于新数据集搜索时过滤低质量架构，显著加快搜索速度并提升最终性能；iii）代理模型可直接作为搜索目标，实现巨大提速。
原文摘要 · Abstract (English)
Neural architecture search (NAS) faces a challenge in balancing the exploration of expressive, broad search spaces that enable architectural innovation with the need for efficient evaluation of architectures to effectively search such spaces. We investigate surrogate model training for improving search in highly expressive NAS search spaces based on context-free grammars. We show that i) surrogate models trained either using zero-cost-proxy metrics and neural graph features (GRAF) or by fine-tuning an off-the-shelf LM have high predictive power for the performance of architectures both within and across datasets, ii) these surrogates can be used to filter out bad architectures when searching on novel datasets, thereby significantly speeding up search and achieving better final performances, and iii) the surrogates can be further used directly as the search objective for huge speed-ups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。