arXiv:2508.19352cs.LG2025-08NeurIPS被引 2

揭示图神经网络记忆训练数据的机制,提出通过重连图结构降低记忆风险。

Memorization in Graph Neural Networks

  • 首次量化节点分类中的标签记忆,发现低同质性图更易引发记忆
  • 低同质性下模型依赖记忆标签以降低损失,与图结构信息不足相关
  • 重连图结构可减少记忆且不损性能,提升隐私安全性

深度神经网络会记忆训练数据,但图神经网络(GNN)的类似分析仍不充分。本文提出NCMemo框架,首次量化半监督节点分类中的标签记忆现象。研究发现,记忆与图同质性呈反比关系:同质性越低,记忆越严重,表明GNN在非同质图上依赖记忆来学习。进一步分析训练动态显示,低同质性下图结构信息不足,导致模型产生对图结构的隐式偏差,从而诱发标签记忆以最小化训练损失。此外,特征空间邻域标签不一致度高的节点更易被记忆。基于此,本文探索图重连作为缓解记忆的方法,结果表明该策略能有效降低记忆程度而不影响模型性能,并显著降低已有数据点的隐私泄露风险。本工作深化了对GNN学习机制的理解,推动更隐私安全的GNN应用。

原文摘要 · Abstract (English)

Deep neural networks (DNNs) have been shown to memorize their training data, yet similar analyses for graph neural networks (GNNs) remain largely under-explored. We introduce NCMemo (Node Classification Memorization), the first framework to quantify label memorization in semi-supervised node classification. We first establish an inverse relationship between memorization and graph homophily, i.e., the property that connected nodes share similar labels/features. We find that lower homophily significantly increases memorization, indicating that GNNs rely on memorization to learn less homophilic graphs. Secondly, we analyze GNN training dynamics. We find that the increased memorization in low homophily graphs is tightly coupled to the GNNs' implicit bias on using graph structure during learning. In low homophily regimes, this structure is less informative, hence inducing memorization of the node labels to minimize training loss. Finally, we show that nodes with higher label inconsistency in their feature-space neighborhood are significantly more prone to memorization. Building on our insights into the link between graph homophily and memorization, we investigate graph rewiring as a means to mitigate memorization. Our results demonstrate that this approach effectively reduces memorization without compromising model performance. Moreover, we show that it lowers the privacy risk for previously memorized data points in practice. Thus, our work not only advances understanding of GNN learning but also supports more privacy-preserving GNN deployment.

图神经网络模型记忆隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。