用基因序列比对算法加速神经网络架构搜索，提升效率与多样性。
Evolutionary Architecture Search through Grammar-Based Sequence Alignment
- 基于基因序列比对思想设计架构距离度量，支持高效交叉操作。
- 计算复杂度显著降低，可快速找到架构间的最短路径。
- 适合追求高效搜索和新架构发现的研究者使用。
在表达能力强的搜索空间中进行神经架构搜索（NAS）是一项计算难题，但也具备自动发现全新且高性能架构的潜力。为此，我们需要高效的搜索算法来识别并复用强大组件。本文引入两种改进的Smith-Waterman算法，用于语法驱动的进化架构搜索中的局部序列比对，以计算架构间的编辑距离。该方法使我们能高效计算架构距离，生成混合后代，并支持基于交叉的搜索启发式策略。它还允许对架构损失曲面进行深入分析，追踪搜索过程中的种群多样性。实验表明，本方法在计算复杂度上远优于先前工作，能够高效计算架构间的最短路径。在进化搜索中实施交叉操作后，性能表现具有竞争力，超越现有方法。未来工作可在此基础上发现更通用的新组件，并拓展至非NAS场景。
原文摘要 · Abstract (English)
Neural architecture search (NAS) in expressive search spaces is a computationally hard problem, but it also holds the potential to automatically discover completely novel and performant architectures. To achieve this we need effective search algorithms that can identify powerful components and reuse them in new candidate architectures. In this paper, we introduce two adapted variants of the Smith-Waterman algorithm for local sequence alignment and use them to compute the edit distance in a grammar-based evolutionary architecture search. These algorithms enable us to efficiently calculate a distance metric for neural architectures and to generate a set of hybrid offspring from two parent models. This facilitates the deployment of crossover-based search heuristics, allows us to perform a thorough analysis on the architectural loss landscape, and track population diversity during search. We highlight how our method vastly improves computational complexity over previous work and enables us to efficiently compute shortest paths between architectures. When instantiating the crossover in evolutionary searches, we achieve competitive results, outperforming competing methods. Future work can build upon this new tool, discovering novel components that can be used more broadly across neural architecture design, and broadening its applications beyond NAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。