提出无需传数据的联邦域泛化方法,通过梯度匹配提升模型跨域适应能力。
Federated Domain Generalization with Data-free On-server Matching Gradient
- 用本地梯度在服务器上寻找跨域不变的梯度方向
- 在4个联邦基准和3个域泛化基准上均超越现有最优方法
- 不增加通信开销,可无缝集成到主流联邦学习框架
域泛化(DG)旨在从多个已知源域中学习一个能在未知目标域上良好泛化的模型。传统DG依赖于训练生成域不变表示的编码器,但在联邦域泛化(FDG)中,各域数据分散在不同客户端,此方法无法适用。本文提出一种新方法——基于服务器端梯度匹配的联邦学习(FedOMG),可高效利用分布式域信息。具体而言,通过最大化梯度内积来寻找所有域间的不变梯度方向。该方法具备双重优势:1)在中心服务器上聚合分布式模型特征,无需额外通信开销;2)与多数现有联邦学习或域泛化方法正交,可无缝集成以进一步提升性能。在多个设置下的广泛实验表明,相比其他联邦/域泛化基线,本方法在四个联邦基准数据集(MNIST、EMNIST、CIFAR-10、CIFAR-100)和三个域泛化基准数据集(PACS、VLCS、OfficeHome)上均取得更优表现。
原文摘要 · Abstract (English)
Domain Generalization (DG) aims to learn from multiple known source domains a model that can generalize well to unknown target domains. One of the key approaches in DG is training an encoder which generates domain-invariant representations. However, this approach is not applicable in Federated Domain Generalization (FDG), where data from various domains are distributed across different clients. In this paper, we introduce a novel approach, dubbed Federated Learning via On-server Matching Gradient (FedOMG), which can \emph{efficiently leverage domain information from distributed domains}. Specifically, we utilize the local gradients as information about the distributed models to find an invariant gradient direction across all domains through gradient inner product maximization. The advantages are two-fold: 1) FedOMG can aggregate the characteristics of distributed models on the centralized server without incurring any additional communication cost, and 2) FedOMG is orthogonal to many existing FL/FDG methods, allowing for additional performance improvements by being seamlessly integrated with them. Extensive experimental evaluations on various settings to demonstrate the robustness of FedOMG compared to other FL/FDG baselines. Our method outperforms recent SOTA baselines on four FL benchmark datasets (MNIST, EMNIST, CIFAR-10, and CIFAR-100), and three FDG benchmark datasets (PACS, VLCS, and OfficeHome).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。