用动力学模型自动调节学习率和动量,让异构联邦学习更快更稳。
Adaptive Federated Learning via Dynamical System Model
- 将联邦学习建模为动力系统,通过临界阻尼思想优化动量参数。
- 仅需一个全局超参数即可自适应调整客户端与服务器的学习率。
- 对超参数不敏感,适合快速原型设计和大规模部署。
异构联邦学习中,客户端计算能力与数据分布差异导致超参数选择至关重要,但手动调参成本高且难以扩展。本文提出一种端到端自适应联邦学习方法,客户端与中心节点可动态调整本地学习率与动量。通过将联邦学习视为动力系统,利用数值模拟中的临界阻尼原理实现稳定快速收敛,同时通过数值精度准则自适应设定学习率。该方法仅依赖单一全局超参数,即可实现客户端与服务器更新的协同自适应,有效缓解目标不一致与客户端漂移问题。实验表明,相比现有先进方法,本框架在异构场景下具有更优收敛性,且无需分别调优客户端与服务器的超参数,显著提升部署效率。
原文摘要 · Abstract (English)
Hyperparameter selection is critical for stable and efficient convergence of heterogeneous federated learning, where clients differ in computational capabilities, and data distributions are non-IID. Tuning hyperparameters is a manual and computationally expensive process as the hyperparameter space grows combinatorially with the number of clients. To address this, we introduce an end-to-end adaptive federated learning method in which both clients and central agents adaptively select their local learning rates and momentum parameters. Our approach models federated learning as a dynamical system, allowing us to draw on principles from numerical simulation and physical design. Through this perspective, selecting momentum parameters equates to critically damping the system for fast, stable convergence, while learning rates for clients and central servers are adaptively selected to satisfy accuracy properties from numerical simulation. The result is an adaptive, momentum-based federated learning algorithm in which the learning rates for clients and servers are dynamically adjusted and controlled by a single, global hyperparameter. By designing a fully integrated solution for both adaptive client updates and central agent aggregation, our method is capable of handling key challenges of heterogeneous federated learning, including objective inconsistency and client drift. Importantly, our approach achieves fast convergence while being insensitive to the choice of the global hyperparameter, making it well-suited for rapid prototyping and scalable deployment. Compared to state-of-the-art adaptive methods, our framework is shown to deliver superior convergence for heterogeneous federated learning while eliminating the need for hyperparameter tuning both client and server updates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。