arXiv:2605.00733cs.NIcs.AI2026-05被引 6

解决联邦多模态学习中遗忘知识纠缠问题,实现精准删除。

EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure

论文配图:EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure
图 1 · 摘自论文原文
  • 通过双分支位移切断跨模态重建通道
  • 利用余弦-正弦分解分离遗忘更新方向,误差低于0.2和4.2
  • 适合需要隐私保护的多模态数据删除场景

联邦多模态学习(FML)在保持图像-文本对隐私的同时,在分布式客户端上训练多模态模型。然而,联合嵌入训练会引发跨模态与客户端梯度子空间的知识纠缠,阻碍联邦遗忘。现有方法既未切断由双线性耦合介导的跨模态重建通道,也未能分离仅针对遗忘客户端的更新方向。我们提出联邦多模态对比遗忘的锚定原则:遗忘对齐通过三类残余锚点持续存在——双线性跨模态耦合、主角子空间纠缠及持续联邦更新。在模态层面,我们证明双向位移视觉与语言分支可关闭跨模态重建通道;对应地,通过客户端更新子空间的余弦-正弦分解,隔离遗忘专属方向。此外,提出方向选择性遗忘锁以限制多轮残留漂移。结合以上策略,提出EASE框架,统一关闭三类锚点通道。EASE在多个数据集与遗忘场景中表现一致领先,例如在Flickr30K上使用CLIP-B/32进行客户端遗忘时,遗忘侧与保留侧分别接近重训练基准,差异仅为0.2和4.2 R@1点。

原文摘要 · Abstract (English)

Federated Multimodal Learning (FML) trains multimodal models across decentralized clients while keeping their image-text pairs private. However, joint embedding training entangles forgotten knowledge across both modalities and client gradient subspaces, hindering federated unlearning. Previous federated unlearning approaches neither sever the cross-modal reconstruction channel mediated by bilinear coupling nor separate forget-exclusive update directions from those shared with retained clients. We identify an Anchor Principle for federated multimodal contrastive unlearning: forgotten alignments persist through three residual anchors arising from bilinear cross-modal coupling, principal-angle subspace entanglement, and continued federated updates. At the modality level, we show that bilateral displacement of both visual and language branches closes the cross-modal reconstruction channel. Correspondingly, our method addresses subspace entanglement through Cosine--Sine decomposition of client-update subspaces, isolating forget-exclusive directions from retain support. Moreover, we propose a direction-selective Forget Lock that bounds residual drift across rounds. Combining these strategies, we present EASE, an Entanglement-Aware Subspace Excision framework that closes all three anchor channels under a unified design. EASE demonstrates consistent superiority across multiple datasets and unlearning scenarios, for instance, matching the retrain reference to within 0.2 and 4.2 R@1 points on the forget and retain sides under client unlearning on Flickr30K with CLIP-B/32.

联邦学习多模态遗忘学习隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。