通过种子架构扩展,高效搜索大规模图神经网络结构。
SA-GNAS: Seed Architecture Expansion for Efficient Large-scale Graph Neural Architecture Search
- 从子图中选出性能一致的种子架构,再迭代扩展提升性能。
- 在含十亿边的大图上比现有方法快2.8倍,且效果更优。
- 适合需要快速设计高性能GNN模型的研究者和工程师。
图神经网络架构搜索(GNAS)在节点分类、链接预测等任务中展现出强大自动设计能力。然而,多数现有GNAS方法因计算与内存开销过大,难以处理百万级节点以上的大型图。为在保持高性能的同时实现大规模图上的高效搜索,本文提出SA-GNAS框架,基于种子架构扩展思想:先构建一个种子架构,再逐步扩展。具体而言,提出基于性能排名一致性的种子架构选择方法,筛选出在子图上表现最接近原大图的架构;并设计基于熵最小化的扩展策略以进一步优化性能。在五个大规模图上的实验表明,SA-GNAS优于人工设计的SOTA GNN架构及现有图NAS方法。此外,其在最大规模图(含十亿边)上相较SOTA方法GAUSS实现2.8倍加速。由于天然可并行,使用更多GPU可进一步提升搜索效率。代码已开源:https://github.com/PasaLab/SAGNAS。
原文摘要 · Abstract (English)
GNAS (Graph Neural Architecture Search) has demonstrated great effectiveness in automatically designing the optimal graph neural architectures for multiple downstream tasks, such as node classification and link prediction. However, most existing GNAS methods cannot efficiently handle large-scale graphs containing more than million-scale nodes and edges due to the expensive computational and memory overhead. To scale GNAS on large graphs while achieving better performance, we propose SA-GNAS, a novel framework based on seed architecture expansion for efficient large-scale GNAS. Similar to the cell expansion in biotechnology, we first construct a seed architecture and then expand the seed architecture iteratively. Specifically, we first propose a performance ranking consistency-based seed architecture selection method, which selects the architecture searched on the subgraph that best matches the original large-scale graph. Then, we propose an entropy minimization-based seed architecture expansion method to further improve the performance of the seed architecture. Extensive experimental results on five large-scale graphs demonstrate that the proposed SA-GNAS outperforms human-designed state-of-the-art GNN architectures and existing graph NAS methods. Moreover, SA-GNAS can significantly reduce the search time, showing better search efficiency. For the largest graph with billion edges, SA-GNAS can achieve 2.8 times speedup compared to the SOTA large-scale GNAS method GAUSS. Additionally, since SA-GNAS is inherently parallelized, the search efficiency can be further improved with more GPUs. SA-GNAS is available at https://github.com/PasaLab/SAGNAS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。