解决联邦学习中跨域数据分布变化问题,提升模型泛化能力。
Unified Alignment Protocol: Making Sense of the Unlabeled Data in New Domains
- 设计交替两阶段训练框架,通过服务器对齐特征分布
- 在多个基准数据集上实现当前最优的跨域泛化性能
- 适合医疗等标签数据稀缺且存在数据分布偏移的场景
半监督联邦学习(SSFL)在现实应用中日益受到关注,因其在客户端标签数据有限的情况下仍能有效训练。传统方法假设训练与测试数据分布一致,但实际中领域偏移频繁发生,亟需提升模型在新未见领域中的泛化能力。本文提出统一对齐协议(UAP),采用交替两阶段训练:第一阶段由服务器学习并用参数化分布对齐特征,随后将该分布传至客户端;第二阶段客户端利用服务器提供的特征分布进行自身特征对齐。大量实验表明,UAP在多个主流域泛化基准数据集及多种模型架构下均达到当前最优性能。
原文摘要 · Abstract (English)
Semi-Supervised Federated Learning (SSFL) is gaining popularity over conventional Federated Learning in many real-world applications. Due to the practical limitation of limited labeled data on the client side, SSFL considers that participating clients train with unlabeled data, and only the central server has the necessary resources to access limited labeled data, making it an ideal fit for real-world applications (e.g., healthcare). However, traditional SSFL assumes that the data distributions in the training phase and testing phase are the same. In practice, however, domain shifts frequently occur, making it essential for SSFL to incorporate generalization capabilities and enhance their practicality. The core challenge is improving model generalization to new, unseen domains while the client participate in SSFL. However, the decentralized setup of SSFL and unsupervised client training necessitates innovation to achieve improved generalization across domains. To achieve this, we propose a novel framework called the Unified Alignment Protocol (UAP), which consists of an alternating two-stage training process. The first stage involves training the server model to learn and align the features with a parametric distribution, which is subsequently communicated to clients without additional communication overhead. The second stage proposes a novel training algorithm that utilizes the server feature distribution to align client features accordingly. Our extensive experiments on standard domain generalization benchmark datasets across multiple model architectures reveal that proposed UAP successfully achieves SOTA generalization performance in SSFL setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。