利用已有图网络知识加速新数据集的模型搜索,提升效率与性能。
Knowledge-aware Evolutionary Graph Neural Architecture Search
- 基于先验知识库构建知识模型与多目标高斯过程联合训练
- 生成候选架构仅需数秒GPU时间,预测准确率高出基线4.27%
- 适合需要快速定制高性能图神经网络的研究者
图神经网络架构搜索(GNAS)可为特定图任务或数据集定制高性能模型。然而现有方法从零知识状态开始搜索,忽略了能提升搜索效率的先验知识。可用的知识库(如NAS-Bench-Graph)包含大量已知架构及其多维度性能指标,如准确率(#Acc)和参数量(#Params)。本文提出知识感知的进化式图神经架构搜索(KEGNAS),利用该先验知识加速在新图数据集上的多目标进化搜索。KEGNAS通过知识库联合训练知识模型与深度多输出高斯过程(DMOGP),可在数秒内生成并评估迁移架构。知识模型建立数据集到架构的映射,快速生成新数据集的候选架构;而结合架构与数据集编码的DMOGP可预测候选架构在新数据集上的多个性能指标。根据预测结果,选取非支配候选架构以热启动多目标进化算法,优化#Acc与#Params。在NAS-Bench-Graph及五个真实数据集上的实证研究表明,KEGNAS能迅速生成高性能架构,准确率比先进进化基线高4.27%,比先进可微基线高11.54%。消融实验表明,使用先验知识显著提升搜索性能。
原文摘要 · Abstract (English)
Graph neural architecture search (GNAS) can customize high-performance graph neural network architectures for specific graph tasks or datasets. However, existing GNAS methods begin searching for architectures from a zero-knowledge state, ignoring the prior knowledge that may improve the search efficiency. The available knowledge base (e.g. NAS-Bench-Graph) contains many rich architectures and their multiple performance metrics, such as the accuracy (#Acc) and number of parameters (#Params). This study proposes exploiting such prior knowledge to accelerate the multi-objective evolutionary search on a new graph dataset, named knowledge-aware evolutionary GNAS (KEGNAS). KEGNAS employs the knowledge base to train a knowledge model and a deep multi-output Gaussian process (DMOGP) in one go, which generates and evaluates transfer architectures in only a few GPU seconds. The knowledge model first establishes a dataset-to-architecture mapping, which can quickly generate candidate transfer architectures for a new dataset. Subsequently, the DMOGP with architecture and dataset encodings is designed to predict multiple performance metrics for candidate transfer architectures on the new dataset. According to the predicted metrics, non-dominated candidate transfer architectures are selected to warm-start the multi-objective evolutionary algorithm for optimizing the #Acc and #Params on a new dataset. Empirical studies on NAS-Bench-Graph and five real-world datasets show that KEGNAS swiftly generates top-performance architectures, achieving 4.27% higher accuracy than advanced evolutionary baselines and 11.54% higher accuracy than advanced differentiable baselines. In addition, ablation studies demonstrate that the use of prior knowledge significantly improves the search performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。