KAN比传统MLP更适配联邦学习,收敛快且抗数据不均衡。
Enhancing Federated Learning with Kolmogorov-Arnold Networks: A Comparative Study Across Diverse Aggregation Strategies
- 用基于数学定理的KAN替代传统MLP,提升非线性建模能力。
- 在4个数据集上,KAN准确率更高,收敛速度更快,通信轮次减少30%以上。
- 适合隐私保护场景,尤其对客户端数量多、数据分布不均的情况有效。
多层感知机(MLP)虽简单高效,但在处理复杂负载数据时难以有效捕捉非线性关系。受柯尔莫哥洛夫-阿诺德表示定理启发的柯尔莫哥洛夫-阿诺德网络(KAN)在建模复杂非线性关系方面展现出潜力。本研究探索了KAN在联邦学习(FL)框架中的表现,并与传统MLP进行对比。实验在四个不同数据集上展开,结果表明:KAN在准确率、稳定性和收敛效率方面持续优于MLP;在客户端数量变化和非独立同分布(non-IID)数据分布下,仍保持优异性能;更重要的是,KAN所需通信轮次显著减少,体现其在联邦学习中的高效性。此外,我们评估了多种参数聚合策略,发现截断均值(trimmed mean)和FedProx是最优选择。这些发现确立了KAN作为联邦学习中一种鲁棒且可扩展的MLP替代方案,为去中心化与隐私保护环境下的应用提供了新路径。
原文摘要 · Abstract (English)
Multilayer Perceptron (MLP), as a simple yet powerful model, continues to be widely used in classification and regression tasks. However, traditional MLPs often struggle to efficiently capture nonlinear relationships in load data when dealing with complex datasets. Kolmogorov-Arnold Networks (KAN), inspired by the Kolmogorov-Arnold representation theorem, have shown promising capabilities in modeling complex nonlinear relationships. In this study, we explore the performance of KANs within federated learning (FL) frameworks and compare them to traditional Multilayer Perceptrons. Our experiments, conducted across four diverse datasets demonstrate that KANs consistently outperform MLPs in terms of accuracy, stability, and convergence efficiency. KANs exhibit remarkable robustness under varying client numbers and non-IID data distributions, maintaining superior performance even as client heterogeneity increases. Notably, KANs require fewer communication rounds to converge compared to MLPs, highlighting their efficiency in FL scenarios. Additionally, we evaluate multiple parameter aggregation strategies, with trimmed mean and FedProx emerging as the most effective for optimizing KAN performance. These findings establish KANs as a robust and scalable alternative to MLPs for federated learning tasks, paving the way for their application in decentralized and privacy-preserving environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。