FedQuad通过四元组学习缓解联邦学习中的数据异质性问题。
FedQuad: Federated Stochastic Quadruplet Learning to Mitigate Data Heterogeneity
- 设计四元组损失函数，缩小类内距离、扩大类间距离。
- 在CIFAR-10和CIFAR-100上优于现有方法，尤其在小数据和不平衡场景下。
- 适合处理数据异构的联邦学习任务，对特征表示学习有显著提升。
联邦学习（FL）实现分布式模型训练，有效解决数据分散与隐私保护问题。然而，客户端间的数据异质性常导致全局模型泛化能力下降，尤其在数据量有限且类别不平衡时更为突出。为此，我们提出新方法FedQuad，显式优化各客户端间的类内方差更小、类间方差更大，从而降低模型聚合对全局表示的负面影响。该方法通过最小化相似样本对的距离、最大化负样本对的距离，在共享特征空间中有效分离客户端数据。我们在多种数据分布和多客户端设置下的CIFAR-10与CIFAR-100数据集上验证了该方法，结果表明其性能优于现有方法。此外，我们还详细分析了基于度量学习的策略在监督与联邦学习范式中的有效性，凸显其在联邦表示学习中的价值。
原文摘要 · Abstract (English)
Federated Learning (FL) provides decentralised model training, which effectively tackles problems such as distributed data and privacy preservation. However, the generalisation of global models frequently faces challenges from data heterogeneity among clients. This challenge becomes even more pronounced when datasets are limited in size and class imbalance. To address data heterogeneity, we propose a novel method, \textit{FedQuad}, that explicitly optimises smaller intra-class variance and larger inter-class variance across clients, thereby decreasing the negative impact of model aggregation on the global model over client representations. Our approach minimises the distance between similar pairs while maximising the distance between negative pairs, effectively disentangling client data in the shared feature space. We evaluate our method on the CIFAR-10 and CIFAR-100 datasets under various data distributions and with many clients, demonstrating superior performance compared to existing approaches. Furthermore, we provide a detailed analysis of metric learning-based strategies within both supervised and federated learning paradigms, highlighting their efficacy in addressing representational learning challenges in federated settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。