用空中聚合与稀疏海森矩阵，大幅降低二阶联邦学习通信开销。
Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation
- 通过稀疏海森估计和空中聚合，减少通信与存储负担。
- 相比基线方法，通信资源与能耗降低超67%。
- 适合大规模模型的高效联邦学习，尤其关注资源受限场景。
二阶联邦学习算法通过利用曲率信息，收敛速度优于一阶方法，但面临高计算与存储成本，尤其在大模型场景下。此外，大模型与数字传输带来的通信开销进一步加剧瓶颈。本文提出一种基于稀疏海森估计与空中聚合的可扩展二阶联邦学习算法，使大模型应用成为可能。仿真结果表明，该方法相比其他一阶与二阶基线，通信资源与能耗节省超过67%。
原文摘要 · Abstract (English)
Second-order federated learning (FL) algorithms offer faster convergence than their first-order counterparts by leveraging curvature information. However, they are hindered by high computational and storage costs, particularly for large-scale models. Furthermore, the communication overhead associated with large models and digital transmission exacerbates these challenges, causing communication bottlenecks. In this work, we propose a scalable second-order FL algorithm using a sparse Hessian estimate and leveraging over-the-air aggregation, making it feasible for larger models. Our simulation results demonstrate more than $67\%$ of communication resources and energy savings compared to other first and second-order baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。