arXiv:2410.02541cs.LGcs.DC2024-10被引 2

针对数据异构性,提出去中心化公平学习算法Facade,提升模型准确率与公平性。

Fair Decentralized Learning

  • 基于特征相似性动态聚类节点,无需预知归属
  • 各聚类独立训练专用模型,实现去中心化协同
  • 在CIFAR-10上比最优基线降低32.3%通信开销

去中心化学习(DL)使节点在不共享原始数据的情况下协作训练模型。但在医疗等场景中,数据特征高度异质会降低模型效用并影响公平性,尤其对少数类数据节点不利。本文提出面向公平训练的聚类型算法Facade,根据本地数据特征相似性动态将节点分配至对应特征簇,且无需节点事先知晓所属簇。Facade(1)随时间动态调整节点归属;(2)在完全去中心化下为每个簇协同训练专用模型。我们理论证明其收敛性,并在三个数据集上对比三种先进基线。实验表明,Facade在准确率和公平性上均优于所有基线。在特征簇大小不平衡时,于CIFAR-10上相较最佳基线减少32.3%通信成本以达目标准确率。

原文摘要 · Abstract (English)

Decentralized learning (DL) is an emerging approach that enables nodes to collaboratively train a machine learning model without sharing raw data. In many application domains, such as healthcare, this approach faces challenges due to the high level of heterogeneity in the training data's feature space. Such feature heterogeneity lowers model utility and negatively impacts fairness, particularly for nodes with under-represented training data. In this paper, we introduce \textsc{Facade}, a clustering-based DL algorithm specifically designed for fair model training when the training data exhibits several distinct features. The challenge of \textsc{Facade} is to assign nodes to clusters, one for each feature, based on the similarity in the features of their local data, without requiring individual nodes to know apriori which cluster they belong to. \textsc{Facade} (1) dynamically assigns nodes to their appropriate clusters over time, and (2) enables nodes to collaboratively train a specialized model for each cluster in a fully decentralized manner. We theoretically prove the convergence of \textsc{Facade}, implement our algorithm, and compare it against three state-of-the-art baselines. Our experimental results on three datasets demonstrate the superiority of our approach in terms of model accuracy and fairness compared to all three competitors. Compared to the best-performing baseline, \textsc{Facade} on the CIFAR-10 dataset also reduces communication costs by 32.3\% to reach a target accuracy when cluster sizes are imbalanced.

去中心化学习公平性聚类联邦学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。