通过梯度范数阈值筛选客户端,提升低碳联邦学习的稳定性和模型性能。
Noise-aware Client Selection for carbon-efficient Federated Learning via Gradient Norm Thresholding
- 引入探针轮次检测客户端数据噪声,用梯度范数筛选优质参与者。
- 实验证明传统方法易选噪声数据,导致模型性能下降。
- 适合关注绿色AI、低碳训练与隐私保护的科研与工程人员。
大规模神经网络训练需大量计算与能源。联邦学习通过分布式数据训练,利用可再生能源降低人工智能训练的碳足迹。已有客户端选择策略试图将可再生能源波动性与稳定公平的模型训练对齐。然而,由于联邦学习的隐私保护特性,客户端数据质量未知,影响有效训练。本文提出一种模块化方法,改进现有低碳联邦学习的客户端选择策略。通过引入噪声客户端过滤机制,增强系统鲁棒性,在数据质量未知场景下同时提升模型性能与可持续性。我们还研究了碳预算对模型收敛的影响,实现效率与可持续性的平衡。大量实验表明,基于本地损失的现代客户端选择策略倾向于选取噪声数据,最终损害模型表现。为此,我们提出基于探针轮次的梯度范数阈值机制,实现更有效的客户端选择与噪声检测,推动低碳联邦学习的实际部署。
原文摘要 · Abstract (English)
Training large-scale Neural Networks requires substantial computational power and energy. Federated Learning enables distributed model training across geospatially distributed data centers, leveraging renewable energy sources to reduce the carbon footprint of AI training. Various client selection strategies have been developed to align the volatility of renewable energy with stable and fair model training in a federated system. However, due to the privacy-preserving nature of Federated Learning, the quality of data on client devices remains unknown, posing challenges for effective model training. In this paper, we introduce a modular approach on top to state-of-the-art client selection strategies for carbon-efficient Federated Learning. Our method enhances robustness by incorporating a noisy client data filtering, improving both model performance and sustainability in scenarios with unknown data quality. Additionally, we explore the impact of carbon budgets on model convergence, balancing efficiency and sustainability. Through extensive evaluations, we demonstrate that modern client selection strategies based on local client loss tend to select clients with noisy data, ultimately degrading model performance. To address this, we propose a gradient norm thresholding mechanism using probing rounds for more effective client selection and noise detection, contributing to the practical deployment of carbon-efficient Federated Learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。