通过选关键数据+部分模型微调,让边缘设备训练快3倍、只用一半数据
Federated Learning with Workload Reduction through Partial Training of Client Models and Entropy-Based Data Selection
- 只训练部分模型参数并选最有信息量的数据,降低边缘设备负担
- 仅用50%数据就超越基线,客户端训练时间减少至1/3
- 适合资源受限的物联网设备,提升联邦学习可扩展性
随着物联网等边缘设备快速普及,其生成的敏感数据亟需参与隐私保护的联邦学习(FL)。现有方法多通过轻量化模型缓解计算压力,但较少关注训练数据量的优化。本文提出FedFT-EDS,结合部分模型微调与基于熵的数据选择,在不牺牲性能前提下显著降低边缘设备的训练负载。通过主动选取最具信息量的本地样本,该方法证明并非所有用户数据在每轮中都同等重要。在CIFAR-10和CIFAR-100上的实验表明,FedFT-EDS仅使用50%的用户数据,即可实现优于基线方法FedAvg和FedProx的全局模型性能;同时,客户端学习效率提升达3倍,训练时间缩短至三分之一即可达到相同效果。本工作凸显了数据选择在联邦学习中的关键作用,为高效可扩展的联邦学习提供了新路径。
原文摘要 · Abstract (English)
With the rapid expansion of edge devices, such as IoT devices, where crucial data needed for machine learning applications is generated, it becomes essential to promote their participation in privacy-preserving Federated Learning (FL) systems. The best way to achieve this desiderate is by reducing their training workload to match their constrained computational resources. While prior FL research has address the workload constrains by introducing lightweight models on the edge, limited attention has been given to optimizing on-device training efficiency through reducing the amount of data need during training. In this work, we propose FedFT-EDS, a novel approach that combines Fine-Tuning of partial client models with Entropy-based Data Selection to reduce training workloads on edge devices. By actively selecting the most informative local instances for learning, FedFT-EDS reduces training data significantly in FL and demonstrates that not all user data is equally beneficial for FL on all rounds. Our experiments on CIFAR-10 and CIFAR-100 show that FedFT-EDS uses only 50% user data while improving the global model performance compared to baseline methods, FedAvg and FedProx. Importantly, FedFT-EDS improves client learning efficiency by up to 3 times, using one third of training time on clients to achieve an equivalent performance to the baselines. This work highlights the importance of data selection in FL and presents a promising pathway to scalable and efficient Federate Learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。