用詹姆斯-斯坦因估计提升跨域联邦学习性能，精度最高提升14%。
FedStein: Enhancing Multi-Domain Federated Learning Through James-Stein Estimator
- 仅共享批量归一化统计量的詹姆斯-斯坦因估计值，保留本地参数不变
- 在三个数据集上验证，部分领域准确率提升超14%且泛化能力更强
- 适合处理数据分布差异大的跨域联邦学习场景
联邦学习（FL）通过在分布式客户端间协作训练实现数据隐私保护。然而，在非独立同分布（non-i.i.d.）数据下，尤其面对来自不同领域、特征分布各异的多域联邦学习时，其性能与收敛性面临严峻挑战。现有研究多关注标签分布偏斜问题，本工作聚焦于较少被探讨的多域异构问题。本文提出新方法 FedStein：通过詹姆斯-斯坦因（James-Stein, JS）估计器共享批量归一化（BN）统计量，同时保留各客户端本地的 BN 参数；非 BN 层参数仍采用标准联邦学习方式交换。在三个数据集及多种模型上的实验表明，FedStein 显著优于 FedAvg、FedBN 等现有方法，在某些领域准确率提升超过 14%，有效增强模型的跨域泛化能力。代码已开源于 https://github.com/sunnyinAI/FedStein。
原文摘要 · Abstract (English)
Federated Learning (FL) facilitates data privacy by enabling collaborative in-situ training across decentralized clients. Despite its inherent advantages, FL faces significant challenges of performance and convergence when dealing with data that is not independently and identically distributed (non-i.i.d.). While previous research has primarily addressed the issue of skewed label distribution across clients, this study focuses on the less explored challenge of multi-domain FL, where client data originates from distinct domains with varying feature distributions. We introduce a novel method designed to address these challenges FedStein: Enhancing Multi-Domain Federated Learning Through the James-Stein Estimator. FedStein uniquely shares only the James-Stein (JS) estimates of batch normalization (BN) statistics across clients, while maintaining local BN parameters. The non-BN layer parameters are exchanged via standard FL techniques. Extensive experiments conducted across three datasets and multiple models demonstrate that FedStein surpasses existing methods such as FedAvg and FedBN, with accuracy improvements exceeding 14% in certain domains leading to enhanced domain generalization. The code is available at https://github.com/sunnyinAI/FedStein
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。