提出梯度重连方法,让图神经网络编辑时不影响其他节点性能
Gradient Rewiring for Editable Graph Neural Network Training
- 用锚定梯度保存训练节点信息,重连目标节点梯度
- 在多个数据集上实现精准编辑,训练节点准确率不下降
- 适合需要局部更新的图神经网络应用,如社交网络维护
深度神经网络广泛应用于计算机视觉、自然语言处理和图分析等领域。然而,模型部署后因环境变化可能产生预测错误。模型编辑旨在用较少数据和计算资源修正错误。尽管视觉与自然语言领域已有进展,图神经网络的可编辑训练仍鲜有研究。其难点在于邻居信息聚合导致编辑影响其他节点。本文首次观察到目标节点与训练节点损失梯度存在显著不一致,直接微调会损害训练节点性能。为此,提出简单有效的梯度重连方法 GRE:先存储训练节点的锚定梯度以保持局部性,再通过锚定梯度重连目标节点梯度,从而保护训练节点表现。实验表明,GRE 在多种架构与图数据集上均有效,适用于多类编辑场景。代码已开源。
原文摘要 · Abstract (English)
Deep neural networks are ubiquitously adopted in many applications, such as computer vision, natural language processing, and graph analytics. However, well-trained neural networks can make prediction errors after deployment as the world changes. \textit{Model editing} involves updating the base model to correct prediction errors with less accessible training data and computational resources. Despite recent advances in model editors in computer vision and natural language processing, editable training in graph neural networks (GNNs) is rarely explored. The challenge with editable GNN training lies in the inherent information aggregation across neighbors, which can lead model editors to affect the predictions of other nodes unintentionally. In this paper, we first observe the gradient of cross-entropy loss for the target node and training nodes with significant inconsistency, which indicates that directly fine-tuning the base model using the loss on the target node deteriorates the performance on training nodes. Motivated by the gradient inconsistency observation, we propose a simple yet effective \underline{G}radient \underline{R}ewiring method for \underline{E}ditable graph neural network training, named \textbf{GRE}. Specifically, we first store the anchor gradient of the loss on training nodes to preserve the locality. Subsequently, we rewire the gradient of the loss on the target node to preserve performance on the training node using anchor gradient. Experiments demonstrate the effectiveness of GRE on various model architectures and graph datasets in terms of multiple editing situations. The source code is available at \url{https://github.com/zhimengj0326/Gradient_rewiring_editing}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。