提出高效精准的联邦学习数据删除方法,支持隐私合规且不重训模型。
Causal Unlearning in Collaborative Optimization: Exact and Approximate Influence Reversal under Adversarial Contributions
- 用克雷洛夫子空间迭代近似影响函数,复杂度从O(d³)降至O(kd)
- 在CIFAR-10上速度提升47.75倍,精度损失小于0.6%,攻击成功率接近重训模型
- 仅更新含被删数据的客户端,保证可解释性与系统稳定性
联邦学习需响应数据删除请求以符合隐私法规,但每次删除后重新训练计算成本过高。本文提出HF-KCU方法,通过克雷洛夫子空间中的共轭梯度迭代近似影响函数,将复杂度从O(d³)降低至O(kd),其中k<<d。因果加权机制确保仅持有被删数据的客户端接收参数更新,防止无关客户端产生虚假变化。该方法能应对有界对抗性扰动,具备现实威胁下的渐进退化能力。我们在ResNet-18、SimpleCNN和ViT-Lite上验证了HF-KCU在CIFAR-10、MNIST和Fashion-MNIST上的表现。在Dirichlet(alpha=0.5)划分下,于CIFAR-10上实现47.75倍加速,测试精度保持在71.16%(基准71.76%),误差低于0.60%。对遗忘集的成员推断攻击成功率仅为0.499,与重训模型一致,证实隐私恢复有效。我们提供收敛性保证,表明克雷洛夫近似误差为O((k¹/²−1)/(k¹/²+1)),其中k为海森矩阵条件数。因果加权机制实现精准更新,仅修改含被删数据的客户端,维护未受影响参与者模型质量,并避免异步场景中基于梯度方法的不稳定性。此设计具有可解释性,每个更新均可追溯至被删数据的影响。方法高效精确,适用于异步删除请求频繁、计算资源受限的生产级联邦系统。
原文摘要 · Abstract (English)
Federated learning systems must support data deletion requests to comply with privacy regulations, yet retraining from scratch after each deletion is computationally prohibitive. We present HF-KCU, a method that removes a client's contribution by approximating the influence function through conjugate gradient iterations in Krylov subspaces, reducing complexity from O(d^3) to O(kd) where k<<d.A causal weighting mechanism ensures that only clients holding the deleted data receive parameter updates, preventing spurious changes to unaffected clients. Our method is designed to handle bounded adversarial perturbations to the Hessian and gradient, providing graceful degradation under realistic threat models. We validate HF-KCU across convolutional (ResNet-18, SimpleCNN) and transformer (ViT-Lite) architectures on CIFAR-10, MNIST, and Fashion-MNIST. On CIFAR-10 under Dirichlet (alpha=0.5) partitioning, HF-KCU achieves 47.75 times speedup over retraining while maintaining test accuracy within 0.60% of the rational baseline(71.16 vs 71.76 %). Membership inference attacks on the forget set yield success rates of 0.499 matching the retrained model and confirming effective privacy restoration. We provide convergence guarantees showing that the Krylov approximation error decreases as O((k ^1/2-1)/(k^1/2+1)) where k is the Hessian condition number. The causal weighting mechanism ensures surgical updates, where only clients holding deleted data are modified, preserving model quality for unaffected participants and avoiding the instability of gradient-based approaches in asynchronous federated settings. This design provides interpretability as each update is directly traceable to the influence of the deleted data. The method's efficiency and precision make it suitable for production federated systems where deletion requests arrive asynchronously and computational budgets are constrained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。