针对异构数据和设备,提出自适应蒸馏与激励机制,提升联邦学习效率与参与度。
Adaptive Dual-Mode Distillation with Incentive Schemes for Scalable, Heterogeneous Federated Learning on Non-IID Data
- 根据客户端能力动态选择模型训练模式,实现异构环境下的高效学习。
- 在非独立同分布数据下,模型准确率提升225%,通信成本显著降低。
- 引入激励机制促进客户端参与,适合资源不均的现实联邦学习场景。
联邦学习(FL)作为一种去中心化学习方法,可在不泄露用户隐私的前提下利用分布式数据。然而,现有方法面临三大挑战:首先,假设所有客户端具备相同建模能力,但实际中因业务需求与算力差异难以满足;其次,统计异构性(即非独立同分布数据)严重影响全局模型性能;第三,需设计低成本激励机制以鼓励客户端参与。为此,本文提出三种方法:DL-SH,支持在统计异构下高效、隐私保护且通信高效的训练;DL-MH,用于处理完全异构模型并缓解统计差异;I-DL-MH,是基于激励的DL-MH扩展,在复杂联邦框架中提升客户端参与度。通过多模型架构、多种数据分布(包括IID与多个非IID场景)及多数据集的全面实验验证,结果表明所提方法显著提升准确率并降低通信开销。相比现有最优方法,DL-SH使全局模型准确率提高153%,I-DL-MH在非独立同分布条件下提升达225%。
原文摘要 · Abstract (English)
Federated Learning (FL) has emerged as a promising decentralized learning (DL) approach that enables the use of distributed data without compromising user privacy. However, FL poses several key challenges. First, it is frequently assumed that every client can train the same machine learning models, however, not all clients are able to meet this assumption because of differences in their business needs and computational resources. Second, statistical heterogeneity (a.k.a. non-IID data) poses a major challenge in FL, which can lead to lower global model performance. Third, while addressing these challenges, there is a need for a cost-effective incentive mechanism to encourage clients to participate in FL training. In response to these challenges, we propose several methodologies: DL-SH, which facilitates efficient, privacy-preserving, and communication-efficient learning in the context of statistical heterogeneity; DL-MH, designed to manage fully heterogeneous models while tackling statistical disparities; and I-DL-MH, an incentive-based extension of DL-MH that promotes client engagement in federated learning training by providing incentives within this complex federated learning framework. Comprehensive experiments were carried out to assess the performance and scalability of the proposed approaches across a range of complex experimental settings. This involved utilizing various model architectures, in diverse data distributions, including IID and several non-IID scenarios, as well as multiple datasets. Experimental results demonstrate that the proposed approaches significantly enhance accuracy and decrease communication costs while effectively addressing statistical heterogeneity and model heterogeneity in comparison to existing state-of-the-art approaches and baselines, with DL-SH improving global model accuracy by 153%, and I-DL-MH achieving a 225% improvement under non-IID conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。