通过动态数据队列与熵驱动选择，提升非独立同分布下的联邦学习收敛速度与精度
Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection
- 构建全局数据子集并动态分发，缓解设备间数据分布差异
- 利用数据熵筛选参与训练设备，提升模型收敛效率与准确性
- 在低数据占比下显著优于现有方法，适合边缘计算场景
联邦学习（FL）是一种在边缘设备上分布式协同训练模型的去中心化方法，具有隐私保护、安全合规和成本效益等优势。本文聚焦于解决数据非独立同分布（non-IID）带来的统计复杂性问题，发现当各设备仅包含单一类别数据时，准确率下降可达10%至30%，其根源为权重偏移，由设备级类别分布与总体分布之间的欧氏距离量化，形成偏差项δ_k。为此，提出一种动态数据队列驱动的联邦学习方法（DDFL），在服务器端构建全局数据子集，并动态分配至各设备；同时基于数据熵指标监测每轮训练过程，实现合理设备选择。进一步提供所提方法的收敛性分析，证明其在实际场景中的可行性。实验表明，仅使用10%的全局数据子集，本方法在MNIST上提升约5%准确率，CIFAR-10提升约18%，CIFAR-100提升20%，超越当前最优聚合算法。
原文摘要 · Abstract (English)
Federated Learning (FL) is a decentralized approach for collaborative model training on edge devices. This distributed method of model training offers advantages in privacy, security, regulatory compliance, and cost-efficiency. Our emphasis in this research lies in addressing statistical complexity in FL, especially when the data stored locally across devices is not identically and independently distributed (non-IID). We have observed an accuracy reduction of up to approximately 10\% to 30\%, particularly in skewed scenarios where each edge device trains with only 1 class of data. This reduction is attributed to weight divergence, quantified using the Euclidean distance between device-level class distributions and the population distribution, resulting in a bias term (\(δ_k\)). As a solution, we present a method to improve convergence in FL by creating a global subset of data on the server and dynamically distributing it across devices using a Dynamic Data queue-driven Federated Learning (DDFL). Next, we leverage Data Entropy metrics to observe the process during each training round and enable reasonable device selection for aggregation. Furthermore, we provide a convergence analysis of our proposed DDFL to justify their viability in practical FL scenarios, aiming for better device selection, a non-sub-optimal global model, and faster convergence. We observe that our approach results in a substantial accuracy boost of approximately 5\% for the MNIST dataset, around 18\% for CIFAR-10, and 20\% for CIFAR-100 with a 10\% global subset of data, outperforming the state-of-the-art (SOTA) aggregation algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。