用架构差异预测提升搜索效率，低成本实现高精度神经网络设计
Delta-NAS: Difference of Architecture Encoding for Predictor-based Evolutionary Neural Architecture Search
- 通过预测相似网络的准确率差异，将搜索空间降维
- 计算复杂度从指数级降至线性，样本效率显著提升
- 适合追求高效精准模型搜索的研究者与工程师
神经网络架构搜索（NAS）在特定任务部署的神经网络设计中持续发挥关键作用。当前主流方法面临搜索空间复杂度攀升和计算成本高的挑战，可分为细粒度高成本与粗粒度低成本两类。本文目标是实现低代价下的细粒度搜索。提出通过预测一对相似网络的准确率差异，将问题投影至低维空间。该范式转变使计算复杂度随搜索空间规模由指数级降至线性。论文提供了坚实的数学基础，并在多个主流NAS基准上进行了充分实验。结果表明，所提方法显著优于现有工作，在性能与样本效率方面均有明显提升。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) continues to serve a key roll in the design and development of neural networks for task specific deployment. Modern NAS techniques struggle to deal with ever increasing search space complexity and compute cost constraints. Existing approaches can be categorized into two buckets: fine-grained computational expensive NAS and coarse-grained low cost NAS. Our objective is to craft an algorithm with the capability to perform fine-grain NAS at a low cost. We propose projecting the problem to a lower dimensional space through predicting the difference in accuracy of a pair of similar networks. This paradigm shift allows for reducing computational complexity from exponential down to linear with respect to the size of the search space. We present a strong mathematical foundation for our algorithm in addition to extensive experimental results across a host of common NAS Benchmarks. Our methods significantly out performs existing works achieving better performance coupled with a significantly higher sample efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。