为不同图实例生成定制化提示,提升模型泛化能力。
Instance-Aware Graph Prompt Learning
- 基于轻量架构为每个实例生成动态提示
- 通过可训练码本量化提示,参数少且高效
- 适合需要强泛化能力的图学习任务
图神经网络在图表示学习中占据主导地位,其性能高度依赖端到端的高质量标签。为此,预训练与微调范式被提出以降低标签成本。随后,预训练任务与下游任务之间的差距推动了图提示学习的发展,该方法通过在原始图数据中插入少量参数的图提示来保持竞争力。然而,现有工作大多局限于学习固定的任务特定提示,难以在多样化实例间良好泛化。为此,本文提出实例感知的图提示学习(IA-GPL),旨在为不同输入实例生成个性化提示。该方法使用轻量级架构为每个实例生成中间提示,通过可训练码本向量对提示进行量化,并采用指数移动平均技术确保训练稳定。在多个数据集和设置下的大量实验表明,IA-GPL相较于现有最优基线表现更优。
原文摘要 · Abstract (English)
Graph neural networks stand as the predominant technique for graph representation learning owing to their strong expressive power, yet the performance highly depends on the availability of high-quality labels in an end-to-end manner. Thus the pretraining and fine-tuning paradigm has been proposed to mitigate the label cost issue. Subsequently, the gap between the pretext tasks and downstream tasks has spurred the development of graph prompt learning which inserts a set of graph prompts into the original graph data with minimal parameters while preserving competitive performance. However, the current exploratory works are still limited since they all concentrate on learning fixed task-specific prompts which may not generalize well across the diverse instances that the task comprises. To tackle this challenge, we introduce Instance-Aware Graph Prompt Learning (IA-GPL) in this paper, aiming to generate distinct prompts tailored to different input instances. The process involves generating intermediate prompts for each instance using a lightweight architecture, quantizing these prompts through trainable codebook vectors, and employing the exponential moving average technique to ensure stable training. Extensive experiments conducted on multiple datasets and settings showcase the superior performance of IA-GPL compared to state-of-the-art baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。