用最优传输对齐联邦学习数据分布，提升模型收敛速度与泛化能力。
Optimal Transport-based Domain Alignment as a Preprocessing Step for Federated Learning
- 基于沃尔什斯坦均值的跨设备数据分布对齐方法
- 在CIFAR-10上减少通信轮次，提升泛化性能
- 适合数据异构严重的联邦学习场景
联邦学习（FL）避免本地数据共享至中心服务器，增强隐私与可扩展性。但数据无法聚合导致标签分布不均衡问题：网络中各设备对目标标签的代表性不均。在非均衡数据上融合本地训练模型会损害全局模型聚合效果，降低本地模型更新质量与分布式代理决策准确率。本文提出一种基于最优传输的预处理算法，通过计算通道维度上的沃尔什斯坦均值，在可信中心服务器上构建目标RGB空间，将各设备数据投影至该空间以最小化全局分布差异，从而降低样本方差，促进学习过程。在CIFAR-10数据集上验证，该方法可在更少通信轮次内实现更高泛化性能。
原文摘要 · Abstract (English)
Federated learning (FL) is a subfield of machine learning that avoids sharing local data with a central server, which can enhance privacy and scalability. The inability to consolidate data leads to a unique problem called dataset imbalance, where agents in a network do not have equal representation of the labels one is trying to learn to predict. In FL, fusing locally-trained models with unbalanced datasets may deteriorate the performance of global model aggregation, and reduce the quality of updated local models and the accuracy of the distributed agents' decisions. In this work, we introduce an Optimal Transport-based preprocessing algorithm that aligns the datasets by minimizing the distributional discrepancy of data along the edge devices. We accomplish this by leveraging Wasserstein barycenters when computing channel-wise averages. These barycenters are collected in a trusted central server where they collectively generate a target RGB space. By projecting our dataset towards this target space, we minimize the distributional discrepancy on a global level, which facilitates the learning process due to a minimization of variance across the samples. We demonstrate the capabilities of the proposed approach over the CIFAR-10 dataset, where we show its capability of reaching higher degrees of generalization in fewer communication rounds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。