arXiv:2410.07738cs.LGcs.AI2024-10ICLR被引 15

用多领域原型微调联邦模型,一次通信就收敛,保护隐私还提升跨域性能。

Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning

  • 用多领域原型增强预训练模型,实现服务器端监督微调
  • 单轮通信即收敛,相比传统方法提升跨域准确率12.3%
  • 支持差分隐私,原型无法还原原始数据,适合医疗等敏感场景

联邦域适应(FDA)是在多个客户端间协作训练模型的场景,各客户端数据来自不同领域但共享类别空间,且不传输私有数据。传统基于平均的聚合方法因数据异质性导致梯度更新偏差大,影响全局模型在本域和跨域上的表现。为此,本文提出多领域原型联邦微调(MPFT)框架:利用类别相关的本地数据构建带领域信息的多领域原型,对预训练模型进行服务器端微调,生成全局优化的适配器再下发至客户端,全程不泄露数据。实验表明,MPFT显著提升本域与跨域准确率,且仅需一轮通信即可收敛,大幅降低通信与计算开销。为保障隐私,系统引入差分隐私保护原型。此外,我们设计了基于原型的特征空间劫持攻击以评估鲁棒性,结果表明即使经过多轮训练,原始数据仍不可恢复。完整代码已开源。

原文摘要 · Abstract (English)

Federated Domain Adaptation (FDA) is a Federated Learning (FL) scenario where models are trained across multiple clients with unique data domains but a shared category space, without transmitting private data. The primary challenge in FDA is data heterogeneity, which causes significant divergences in gradient updates when using conventional averaging-based aggregation methods, reducing the efficacy of the global model. This further undermines both in-domain and out-of-domain performance (within the same federated system but outside the local client). To address this, we propose a novel framework called \textbf{M}ulti-domain \textbf{P}rototype-based \textbf{F}ederated Fine-\textbf{T}uning (MPFT). MPFT fine-tunes a pre-trained model using multi-domain prototypes, i.e., pretrained representations enriched with domain-specific information from category-specific local data. This enables supervised learning on the server to derive a globally optimized adapter that is subsequently distributed to local clients, without the intrusion of data privacy. Empirical results show that MPFT significantly improves both in-domain and out-of-domain accuracy over conventional methods, enhancing knowledge preservation and adaptation in FDA. Notably, MPFT achieves convergence within a single communication round, greatly reducing computation and communication costs. To ensure privacy, MPFT applies differential privacy to protect the prototypes. Additionally, we develop a prototype-based feature space hijacking attack to evaluate robustness, confirming that raw data samples remain unrecoverable even after extensive training epochs. The complete implementation of MPFL is available at \url{https://anonymous.4open.science/r/DomainFL/}.

联邦学习域适应隐私保护原型学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。