针对联邦学习中客户端模型修正过矫问题,提出细粒度自适应修正方法。
TACO: Tackling Over-correction in Federated Learning with Tailored Adaptive Correction
- 按客户端差异动态调整梯度修正系数,避免统一修正导致的过矫正。
- 在多个数据集上实现更优且稳定的模型性能,收敛更可靠。
- 轻量级设计,仅需同步模型参数,适合资源受限的边缘计算场景。
边缘客户端间非独立同分布(Non-IID)数据长期制约联邦学习(FL)训练效果。现有方法虽理论表现良好,但首次揭示了因各客户端采用统一模型修正系数而引发的隐性过矫正现象,该现象会损害模型性能甚至导致收敛失败。为此,本文提出TACO算法,通过实施细粒度、客户端定制化的梯度修正与模型聚合策略,引导本地模型向更准确的全局最优解逼近。同时,验证主流FL算法在通信轮次上的准确性优于实际耗时,源于客户端额外计算开销。TACO采用轻量级修正与定制化聚合机制,仅需同步模型参数,无额外信息与计算负担。通过首个揭示过矫正根源的联邦学习收敛分析,结合多数据集广泛实验,证实TACO在实际应用中具备更优且稳定的表现。
原文摘要 · Abstract (English)
Non-independent and identically distributed (Non-IID) data across edge clients have long posed significant challenges to federated learning (FL) training in edge computing environments. Prior works have proposed various methods to mitigate this statistical heterogeneity. While these works can achieve good theoretical performance, in this work we provide the first investigation into a hidden over-correction phenomenon brought by the uniform model correction coefficients across clients adopted by existing methods. Such over-correction could degrade model performance and even cause failures in model convergence. To address this, we propose TACO, a novel algorithm that addresses the non-IID nature of clients' data by implementing fine-grained, client-specific gradient correction and model aggregation, steering local models towards a more accurate global optimum. Moreover, we verify that leading FL algorithms generally have better model accuracy in terms of communication rounds rather than wall-clock time, resulting from their extra computation overhead imposed on clients. To enhance the training efficiency, TACO deploys a lightweight model correction and tailored aggregation approach that requires minimum computation overhead and no extra information beyond the synchronized model parameters. To validate TACO's effectiveness, we present the first FL convergence analysis that reveals the root cause of over-correction. Extensive experiments across various datasets confirm TACO's superior and stable performance in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。