保护隐私的模型遗忘技术，让客户端在不接触原始参数情况下安全删掉数据。
MPU: Towards Secure and Privacy-Preserving Knowledge Unlearning for Large Language Models
- 服务器生成多个带扰动的模型副本，客户端本地执行遗忘操作。
- 实验显示多数算法在10%噪声下性能下降低于1%，部分还优于无噪声基线。
- 适合注重数据隐私与合规性的大模型应用，如医疗、金融领域。
大型语言模型的机器遗忘常面临隐私困境：严格限制服务器参数与客户端遗忘集的共享。为解决这一双重保密约束，我们提出MPU——一种与算法无关的隐私保护多扰动副本遗忘框架，包含两个服务端模块：预处理模块用于随机生成副本，后处理模块实现更新聚合。预处理阶段，服务器分发多个经过扰动和重参数化的模型实例，使客户端能在不访问服务器原始参数的前提下，在私有遗忘集上本地执行遗忘操作。本地遗忘完成后，服务器通过反向重参数化并采用调和去噪方法聚合更新，以缓解扰动影响。七种遗忘算法的实验表明，MPU在性能上可媲美无噪声基线，大多数算法在高达10%噪声下平均退化低于1%，甚至在1%噪声下部分算法表现优于无噪声基线。代码已开源于https://github.com/Tristan0318/MPU。
原文摘要 · Abstract (English)
Machine unlearning for large language models often faces a privacy dilemma in which strict constraints prohibit sharing either the server's parameters or the client's forget set. To address this dual non-disclosure constraint, we propose MPU, an algorithm-agnostic privacy-preserving Multiple Perturbed Copies Unlearning framework that primarily introduces two server-side modules: Pre-Process for randomized copy generation and Post-Process for update aggregation. In Pre-Process, the server distributes multiple perturbed and reparameterized model instances, allowing the client to execute unlearning locally on its private forget set without accessing the server's exact original parameters. After local unlearning, the server performs Post-Process by inverting the reparameterization and aggregating updates with a harmonic denoising procedure to alleviate the impact of perturbation. Experiments with seven unlearning algorithms show that MPU achieves comparable unlearning performance to noise-free baselines, with most algorithms' average degradation well below 1% up to 10% noise, and can even outperform the noise-free baseline for some algorithms under 1% noise. Code is available at https://github.com/Tristan0318/MPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。