统一提示框架让预训练图模型通用化,适应不同场景。
One Prompt Fits All: Universal Graph Adaptation for Pretrained Models
- 提出统一提示方法,聚焦释放预训练模型能力。
- 跨域测试中性能稳定,优于现有多种提示策略。
- 适合需要泛化能力的图学习下游任务使用。
图提示学习(GPL)作为连接预训练图模型与下游任务的新范式,缓解了标签依赖和上下游任务不匹配问题。然而现有研究在提示机制理解上缺乏共识:不同策略作用于输入层、层间或表示层,干预方式不一。同时多数方法难以跨场景泛化,尤其在同质图到异质图的数据分布变化下表现不佳。本文通过理论分析发现,表示层提示本质等价于微调简单分类器,主张应让预训练模型保持原能力,由分类器适配下游场景。基于此,提出UniPrompt,一种可适配任意预训练模型的通用图提示方法,在保留原始图结构的前提下,显著提升模型在同域与跨域任务中的表现。大量实验验证其有效性与普适性。
原文摘要 · Abstract (English)
Graph Prompt Learning (GPL) has emerged as a promising paradigm that bridges graph pretraining models and downstream scenarios, mitigating label dependency and the misalignment between upstream pretraining and downstream tasks. Although existing GPL studies explore various prompt strategies, their effectiveness and underlying principles remain unclear. We identify two critical limitations: (1) Lack of consensus on underlying mechanisms: Despite current GPLs have advanced the field, there is no consensus on how prompts interact with pretrained models, as different strategies intervene at varying spaces within the model, i.e., input-level, layer-wise, and representation-level prompts. (2) Limited scenario adaptability: Most methods fail to generalize across diverse downstream scenarios, especially under data distribution shifts (e.g., homophilic-to-heterophilic graphs). To address these issues, we theoretically analyze existing GPL approaches and reveal that representation-level prompts essentially function as fine-tuning a simple downstream classifier, proposing that graph prompt learning should focus on unleashing the capability of pretrained models, and the classifier should adapt to downstream scenarios. Based on our findings, we propose UniPrompt, a novel GPL method that adapts any pretrained models, unleashing the capability of pretrained models while preserving the input graph. Extensive experiments demonstrate that our method can effectively integrate with various pretrained models and achieve strong performance across in-domain and cross-domain scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。