用神经正切核提升异构环境下的去中心化联邦学习性能
NTK-DFL: Enhancing Decentralized Federated Learning in Heterogeneous Settings via Neural Tangent Kernel
- 结合神经正切核与模型平均，利用客户端间差异提升训练效果
- 在高度异构设置下准确率超越基线，通信轮次减少4.6倍
- 适用于数据分布差异大的实际场景，尤其适合隐私敏感应用
去中心化联邦学习（DFL）是一种无需中央服务器且不交换原始数据的协作式机器学习框架。由于参与者数据分布存在统计异构性（反映本地环境与用户行为差异），DFL面临挑战。已有研究表明，在集中式联邦学习中使用神经正切核（NTK）可提升性能。本文提出一种在去中心化设置下利用NTK训练客户端模型的方法，并引入基于NTK的模型演化与模型平均之间的协同机制。该协同利用客户端间模型偏差，显著提升异构环境下的准确率与收敛速度。实验表明，本方法在高度异构设置下持续优于基线，在多个数据集、网络拓扑和异构度设置下均表现稳健。相比基线，达到目标性能所需通信轮次减少4.6倍。
原文摘要 · Abstract (English)
Decentralized federated learning (DFL) is a collaborative machine learning framework for training a model across participants without a central server or raw data exchange. DFL faces challenges due to statistical heterogeneity, as participants often possess data of different distributions reflecting local environments and user behaviors. Recent work has shown that the neural tangent kernel (NTK) approach, when applied to federated learning in a centralized framework, can lead to improved performance. We propose an approach leveraging the NTK to train client models in the decentralized setting, while introducing a synergy between NTK-based evolution and model averaging. This synergy exploits inter-client model deviation and improves both accuracy and convergence in heterogeneous settings. Empirical results demonstrate that our approach consistently achieves higher accuracy than baselines in highly heterogeneous settings, where other approaches often underperform. Additionally, it reaches target performance in 4.6 times fewer communication rounds. We validate our approach across multiple datasets, network topologies, and heterogeneity settings to ensure robustness and generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。