用专家提示混合框架提升图模型跨任务泛化能力
GMoPE:A Prompt-Expert Mixture Framework for Graph Foundation Models
- 引入专家提示向量与结构感知路由机制,实现专家分工
- 仅需少量参数更新即达全量微调效果,降低适配成本
- 适合需要高效迁移的图神经网络应用开发
图神经网络在特定任务上表现优异,但跨领域、跨任务的泛化能力仍有限。现有方法常面临负迁移、可扩展性差和高适配成本问题。本文提出GMoPE(图专家提示混合框架),将提示学习与专家混合(MoE)架构结合,通过专家专属提示向量和结构感知路由机制,使每个专家专注于特定子域并动态参与预测。为增强多样性、防止专家坍塌,引入提示向量间的软正交约束,促进专家分化与均衡使用。采用仅提示微调策略,显著降低迁移过程中的时空复杂度。在多种预训练策略和下游任务上的实验表明,GMoPE持续优于现有最优基线,在性能上接近全参数微调,而适应开销仅为一小部分。本工作为构建可泛化且高效的图基础模型提供了系统性解决方案。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have demonstrated impressive performance on task-specific benchmarks, yet their ability to generalize across diverse domains and tasks remains limited. Existing approaches often struggle with negative transfer, scalability issues, and high adaptation costs. To address these challenges, we propose GMoPE (Graph Mixture of Prompt-Experts), a novel framework that seamlessly integrates the Mixture-of-Experts (MoE) architecture with prompt-based learning for graphs. GMoPE leverages expert-specific prompt vectors and structure-aware MoE routing to enable each expert to specialize in distinct subdomains and dynamically contribute to predictions. To promote diversity and prevent expert collapse, we introduce a soft orthogonality constraint across prompt vectors, encouraging expert specialization and facilitating a more balanced expert utilization. Additionally, we adopt a prompt-only fine-tuning strategy that significantly reduces spatiotemporal complexity during transfer. We validate GMoPE through extensive experiments under various pretraining strategies and multiple downstream tasks. Results show that GMoPE consistently outperforms state-of-the-art baselines and achieves performance comparable to full parameter fine-tuning-while requiring only a fraction of the adaptation overhead. Our work provides a principled and scalable framework for advancing generalizable and efficient graph foundation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。