通过正交约束提升联邦学习中全局原型的分离度，显著改善异构场景下的模型性能。
FedORGP: Guiding Heterogeneous Federated Learning with Orthogonality Regularization on Global Prototypes
- 引入正交性正则化，增强不同类原型间的角度分离
- 在统计与模型异构共存下，最高提升10.12%准确率
- 适合处理数据分布差异大的联邦学习场景
联邦学习（FL）已成为分布式机器学习的重要框架，尤其在保护隐私的数据处理方面具有潜力。然而，现有框架难以应对统计异构和模型异构问题，严重影响模型表现。虽然异构联邦学习（HtFL）采用基于原型的方法缓解挑战，但当前方法在原型最优分离方面仍存在局限。本文提出FedORGP，一种新型HtFL算法，通过正交性正则化改进全局原型的分离度，不仅促进类内原型相似性，还显著扩大类间夹角分离。在全局原型引导下，各客户端保持嵌入向量与对应原型在特征空间中的对齐，实现方向独立性，并与交叉熵损失无缝结合。我们提供了非凸条件下FedORGP收敛性的理论证明。大量实验表明，FedORGP优于七种先进基线，在统计与模型异构共存场景下，最高实现10.12%的准确率提升。
原文摘要 · Abstract (English)
Federated Learning (FL) has emerged as an essential framework for distributed machine learning, especially with its potential for privacy-preserving data processing. However, existing FL frameworks struggle to address statistical and model heterogeneity, which severely impacts model performance. While Heterogeneous Federated Learning (HtFL) introduces prototype-based strategies to address the challenges, current approaches face limitations in achieving optimal separation of prototypes. This paper presents FedORGP, a novel HtFL algorithm designed to improve global prototype separation through orthogonality regularization, which not only encourages intra-class prototype similarity but also significantly expands the inter-class angular separation. With the guidance of the global prototype, each client keeps its embeddings aligned with the corresponding prototype in the feature space, promoting directional independence that integrates seamlessly with the cross-entropy (CE) loss. We provide theoretical proof of FedORGP's convergence under non-convex conditions. Extensive experiments demonstrate that FedORGP outperforms seven state-of-the-art baselines, achieving up to 10.12\% accuracy improvement in scenarios where statistical and model heterogeneity coexist.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。