用零样本搜索自动剪枝搜索空间，大幅降低显存占用
Half Search Space is All You Need
- 先用零样本方法筛选并移除低效架构
- 在DARTS搜索空间上显存降低81%且精度不变
- 适合追求高效神经网络架构搜索的研究者
神经架构搜索（NAS）是自动化设计网络结构的强大工具。一次性NAS方法（如DARTS）因其搜索效率与实现简便而广受欢迎。然而，这类方法在搜索阶段存在高显存消耗的问题。为此，我们提出一种高效自动剪枝搜索空间的方法，以降低内存占用和搜索时间，同时保持搜索精度。具体而言，我们利用零样本NAS在应用一次性NAS前，高效剔除表现差的架构。在DARTS搜索空间上的实验表明，该方法相比基线一次性设置显存消耗降低81%，同时达到相同的准确率。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) is a powerful tool for automating architecture design. One-Shot NAS techniques, such as DARTS, have gained substantial popularity due to their combination of search efficiency with simplicity of implementation. By design, One-Shot methods have high GPU memory requirements during the search. To mitigate this issue, we propose to prune the search space in an efficient automatic manner to reduce memory consumption and search time while preserving the search accuracy. Specifically, we utilise Zero-Shot NAS to efficiently remove low-performing architectures from the search space before applying One-Shot NAS to the pruned search space. Experimental results on the DARTS search space show that our approach reduces memory consumption by 81% compared to the baseline One-Shot setup while achieving the same level of accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。