从数据操作角度理论解析图提示的有效性,给出可证明的性能边界。
Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis
- 基于数据操作视角建立理论框架,证明图提示可逼近图变换算子。
- 推导单图与批量图上数据操作误差的上界,涵盖线性与非线性模型。
- 为推荐、生物网络等场景提供理论依据,适合研究图模型泛化性的学者。
近年来,图提示作为一种新兴研究方向,可在不重新训练预训练图模型的前提下,通过在原始图上附加额外的节点或子图来学习新信息,广泛应用于推荐系统、生物网络和图迁移等领域。该范式从传统的预训练-微调转向预训练-提示,已在模拟图数据操作方面展现出显著的实证成功。然而,其理论基础仍不充分,关于其有效性为何及多大程度上成立的问题尚未解决。为填补这一空白,本文提出一个理论框架,从数据操作视角严谨分析图提示。主要贡献包括:第一,给出形式化保证定理,证明图提示具备逼近图变换算子的能力,有效连接上游与下游任务;第二,推导出单个图及常见于图模型训练中的批量图上数据操作误差的上界;第三,分析数据操作误差的分布特性,将理论结果从线性图模型(如GCN)扩展至非线性图模型(如GAT)。大量实验验证了理论结果,并确认了这些保证的实际意义。
原文摘要 · Abstract (English)
In recent years, graph prompting has emerged as a promising research direction, enabling the learning of additional tokens or subgraphs appended to the original graphs without requiring retraining of pre-trained graph models across various applications. This novel paradigm, shifting from the traditional pretraining and finetuning to pretraining and prompting has shown significant empirical success in simulating graph data operations, with applications ranging from recommendation systems to biological networks and graph transferring. However, despite its potential, the theoretical underpinnings of graph prompting remain underexplored, raising critical questions about its fundamental effectiveness. The lack of rigorous theoretical proof of why and how much it works is more like a dark cloud over the graph prompt area to go further. To fill this gap, this paper introduces a theoretical framework that rigorously analyzes graph prompting from a data operation perspective. Our contributions are threefold: First, we provide a formal guarantee theorem, demonstrating graph prompts capacity to approximate graph transformation operators, effectively linking upstream and downstream tasks. Second, we derive upper bounds on the error of these data operations by graph prompts for a single graph and extend this discussion to batches of graphs, which are common in graph model training. Third, we analyze the distribution of data operation errors, extending our theoretical findings from linear graph models (e.g., GCN) to non-linear graph models (e.g., GAT). Extensive experiments support our theoretical results and confirm the practical implications of these guarantees.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。