用贝叶斯优化生成图结构,让模型快速适配新任务
Learn to Explore: Meta NAS via Bayesian Optimization Guided Graph Generation
- 将网络结构建模为图,结合贝叶斯优化与梯度上升搜索
- 在多个数据集上超越现有方法,泛化能力更强
- 适合需要快速部署新模型的工程场景
神经架构搜索(NAS)可自动设计高性能神经网络,但通常针对单一预定义任务,限制了实际应用。为此,元神经架构搜索(Meta-NAS)通过利用跨任务先验知识,实现对新任务的快速适应。然而,现有Meta-NAS方法普遍存在泛化能力差、搜索空间有限或计算成本高等问题。本文提出一种新型Meta-NAS框架GraB-NAS:首先将神经网络结构建模为图,再采用混合搜索策略,在搜索空间中通过贝叶斯优化进行全局架构搜索,并在潜在空间中通过梯度上升实现局部探索,以生成性能优异的新架构。该策略使GraB-NAS能在预定义搜索空间外发现任务感知型架构,且具备强性能。大量实验表明,GraB-NAS显著优于当前先进Meta-NAS基线,在泛化性和搜索效率方面表现更优。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) automates the design of high-performing neural networks but typically targets a single predefined task, thereby restricting its real-world applicability. To address this, Meta Neural Architecture Search (Meta-NAS) has emerged as a promising paradigm that leverages prior knowledge across tasks to enable rapid adaptation to new ones. Nevertheless, existing Meta-NAS methods often struggle with poor generalization, limited search spaces, or high computational costs. In this paper, we propose a novel Meta-NAS framework, GraB-NAS. Specifically, GraB-NAS first models neural architectures as graphs, and then a hybrid search strategy is developed to find and generate new graphs that lead to promising neural architectures. The search strategy combines global architecture search via Bayesian Optimization in the search space with local exploration for novel neural networks via gradient ascent in the latent space. Such a hybrid search strategy allows GraB-NAS to discover task-aware architectures with strong performance, even beyond the predefined search space. Extensive experiments demonstrate that GraB-NAS outperforms state-of-the-art Meta-NAS baselines, achieving better generalization and search effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。