提出高效知识图谱遗忘算法，无需重训练即可快速移除数据影响。
Efficient Knowledge Graph Unlearning with Zeroth-order Information
- 基于零阶优化与泰勒展开，避免计算一阶、二阶导数
- 在大规模知识图谱上实现更优的遗忘效率与质量
- 适合需要合规删除数据的工业级知识图谱系统
由于《被遗忘的权利》等法规要求，模型需移除训练数据及其影响。全量重训练成本过高，现有机器遗忘方法难以高效处理知识图谱（KG）结构。本文首次提出高效的KG遗忘算法，指出其挑战源于图结构特性与实体间语义关系。传统基于影响估计的方法在大规模图上计算开销巨大。为此，我们定义了适用于KG的影响力函数，通过泰勒展开估算数据移除后的参数变化，避免昂贵的一阶与二阶导数计算。利用费舍尔矩阵和零阶优化，近似逆海森向量积，无需构建计算图。实验表明，该方法在遗忘效率和质量上显著优于当前最先进基线。代码已开源：https://github.com/NKUShaw/ZOWFKGIF。
原文摘要 · Abstract (English)
Due to regulations like the Right to be Forgotten, there is growing demand for removing training data and its influence from models. Since full retraining is costly, various machine unlearning methods have been proposed. In this paper, we firstly present an efficient knowledge graph (KG) unlearning algorithm. We remark that KG unlearning is nontrivial due to the distinctive structure of KG and the semantic relations between entities. Also, unlearning by estimating the influence of removed components incurs significant computational overhead when applied to large-scale knowledge graphs. To this end, we define an influence function for KG unlearning and propose to approximate the model's sensitivity without expensive computation of first-order and second-order derivatives for parameter updates. Specifically, we use Taylor expansion to estimate the parameter changes caused by data removal. Given that the first-order gradients and second-order derivatives dominate the computational load, we use the Fisher matrices and zeroth-order optimization to approximate the inverse-Hessian vector product without constructing the computational graphs. Our experimental results demonstrate that the proposed method outperforms other state-of-the-art graph unlearning baselines significantly in terms of unlearning efficiency and unlearning quality. Our code is released at https://github.com/NKUShaw/ZOWFKGIF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。