Chimaera用专家混合架构提升图模型跨任务跨数据集泛化能力
Chimaera: A Mixture-of-Graph-Experts Architecture for Cross-Task and Cross-Dataset Graph Learning
- 融合多专家与图基础模型，支持节点、边、图三类任务
- 仅用少量样本即可在6个数据集上实现强迁移性能
- 线性GNN在跨任务中表现优异，需搭配大小语言模型生成嵌入
设计图神经网络的基础模型面临图结构不规则及嵌入维度差异的挑战。Chimaera将专家混合机制与图基础模型（GFM）结合，集成图提示和线性GNN等多种架构。利用大语言模型生成嵌入，专家可按不同策略训练与组合。该方法扩展了现有线性GNN的能力，支持节点、边和图级别任务。在六个文本属性图基准数据集上进行同任务与跨任务实验，涵盖节点、边和图分类任务。实验证明Chimaera有效实现任务与数据集间的迁移。进一步分析表明：需同时使用大模型和小模型生成嵌入；简单有效的线性GNN具备强跨任务迁移能力；仅用少量样本即可获得良好结果。
原文摘要 · Abstract (English)
Designing foundation models for graphs is challenging due to the irregular structure of graphs and the different sizes and characteristics of embeddings. Chimaera integrates mixture-of-experts with graph foundation models (GFM). It integrates different GFM architectures, such as graph prompts and linear GNN models. Large language models are used to generate embeddings, and experts can be trained and combined following different strategies, GFMs, embeddings, etc. Furthermore, Chimaera extends existing linear GNNs to support link-level and graph-level tasks in addition to node-level tasks. Empirical analyses are performed on same-task and cross-task experiments with node, link, and graph classification tasks using six benchmark text-attributed graph datasets. The experiments demonstrate the effectiveness of Chimaera and its capabilities for transfer across tasks and datasets. Further insights include the need to use both large and small language models to generate embeddings for the experts, a strong cross-task transferability of simple but effective linear GNNs, and using few samples only to provide strong results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。