实现冻结模型下精准联邦遗忘，高效删除用户数据
Exact Federated Continual Unlearning for Ridge Heads on Frozen Foundation Models
- 用两个统计量编码数据，通过固定大小消息通信
- 删除请求后模型与集中式重训练完全一致，误差小于10⁻⁹
- 适合需精确删除的隐私敏感场景，如医疗联邦学习
基础模型常以冻结特征提取器形式部署，并搭配可训练头部适应私有用户数据，应用于联邦学习环境。根据“被遗忘权”要求，需按需移除特定样本或用户对模型的影响。现有联邦遗忘方法针对通用深度模型，依赖近似重构或选择性重训练，导致精确性代价高昂或难以实现。本文研究一个实际但未被充分探索的场景：冻结的基础模型搭配岭回归头部。其最优解仅依赖于两个加性充分统计量，我们据此设计通信协议，支持任意流式增删请求，且消息大小固定。服务器维护的头部在精确算术下，每次请求后均与集中式重训练点对点一致。我们提供确定性重训练等价保证、顺序与分组无关性、两种服务器端变体及零KL散度的贝叶斯验证。四个基准测试结果表明：两种变体在相对弗罗贝尼乌斯误差上均达到10⁻⁹以内，且单次请求成本较联邦重训练基线低数个数量级。
原文摘要 · Abstract (English)
Foundation models are commonly deployed as frozen feature extractors with a small trainable head to adapt to private, user-generated data in federated settings. The ``right to be forgotten'' requires removing the influence of specific samples or users from the trained model on demand. Existing federated unlearning methods target general deep models and rely on approximate reconstruction or selective retraining, making exactness costly or elusive. We study this problem in a practically relevant but under-explored regime: a frozen foundation model with a ridge-regression head. The exact optimum depends on the data only through two additive sufficient statistics, which we turn into a communication protocol supporting an arbitrary stream of add and delete requests via fixed-size messages. The server maintains a head that is, in exact arithmetic, pointwise identical to centralized retraining after every request. We provide deterministic retrain-equivalence guarantees, order and partition invariance, two server-side variants, and a Bayesian certificate of zero KL divergence. Experiments on four benchmarks confirm the guarantees: both variants match centralized ridge retraining to within $10^{-9}$ relative Frobenius error and complete each request at orders-of-magnitude lower cost than federated retraining baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。