提出新框架揭示视觉模型在联邦学习中其实没真正遗忘。
Do Vision Models Truly Forget? New Findings from Representation-Level Certification of Visual Unlearning in Vertical Federated Learning

- 从表示层而非输出层检测遗忘,用四种诊断方法评估
- 现有方法通过输出检验却仍保留明显类别结构,最高差15.4分
- 发现类别级遗忘比样本级更难抹除,适合关注隐私安全的研究者
垂直联邦学习中的机器去学习受到越来越多关注,但现有方法仅通过输出层指标验证遗忘。我们提出Mirage——一种表示层审计框架,包含四种互补诊断:线性探测恢复(LPR)、中心核对齐(CKA)、特征可分性评分和逐层恢复分析。在七种数据集和七种基线方法上,遵循最新VFL去学习协议的实验揭示三个关键发现:(1) 遗忘差距:通过输出层认证的方法仍保有显著类别结构,其LPR最高比重新训练基线高出15.4分;CKA显示这些模型在结构上仍更接近原始模型而非重新训练参考模型;可分性评分表明持续存在几何判别能力。(2) 去学习三难困境:无现有方法能同时实现高效用、输出层遗忘和表示层遗忘。(3) 类别-样本不对称性:类别级遗忘留下强烈表示痕迹(某些数据集上LPR超过96%),而样本级遗忘与随机水平无异(LPR约50%);逐层分析进一步表明残留类别信息贯穿网络深层。这些发现呼吁建立更重视表示的联邦去学习评估标准。代码公开于https://github.com/YuZhenyuLindy/Mirage。
原文摘要 · Abstract (English)
Machine unlearning in Vertical Federated Learning (VFL) has attracted growing interest, yet existing methods certify forgetting solely using output-level metrics. We challenge these works by introducing Mirage, a representation-level auditing framework that comprises four complementary diagnostics: Linear probe recovery (LPR), centered kernel alignment (CKA), feature separability scoring, and layer-wise recovery analysis. Extensive experiments across seven datasets and seven baseline methods following recent VFL unlearning protocols reveal three key findings: (1) Forgetting gap: methods that pass output-level certification still retain substantial class structure in their representations, with LPR exceeding the retrained baseline by up to 15.4 points; CKA shows that these models remain structurally closer to the original than to the retrained reference, while separability scores indicate persistent geometric discrimination. (2) Unlearning trilemma: no existing method simultaneously achieves high utility, output-level forgetting, and representation-level forgetting. (3) Class-sample asymmetry: class-level forgetting leaves strong representational traces (LPR exceeding 96 percent on several datasets), whereas sample-level forgetting is indistinguishable from chance (LPR is approximately 50 percent); layer-wise analysis further shows that residual class information persists across network depths. These findings call for representation-aware evaluation standards in federated unlearning research. Code is publicly available at https://github.com/YuZhenyuLindy/Mirage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。