arXiv:2508.04470cs.LG2025-08

提出无需梯度更新的个性化联邦学习方法,有效应对数据异构挑战

FedHiP: Heterogeneity-Invariant Personalized Federated Learning Through Closed-Form Solutions

  • 用闭式解替代梯度更新,避免非独立同分布数据干扰
  • 在多个基准数据集上精度领先基线5.79%-20.97%
  • 适合数据异构严重的个性化模型部署场景

个性化联邦学习(PFL)通过协同训练同时适配客户端本地应用,成为主流范式。现有方法普遍受客户端间数据异构(非独立同分布)严重影响,导致收敛困难、性能下降。本文指出根本原因在于长期依赖对非独立同分布数据敏感的梯度更新。为此,提出新型异构不变个性化联邦学习框架FedHiP,采用解析(即闭式)解避免梯度更新。利用自监督预训练的冻结主干网络进行无梯度特征提取,并设计解析分类器实现无梯度训练。FedHiP包含三个阶段:解析本地训练、解析全局聚合与解析本地个性化。其闭式解赋予理想特性——异构不变性,即无论其他客户端数据如何非独立同分布,个性化模型保持一致。大量实验验证,该方案在基准数据集上精度优于当前最优基线至少5.79%~20.97%。

原文摘要 · Abstract (English)

Lately, Personalized Federated Learning (PFL) has emerged as a prevalent paradigm to deliver personalized models by collaboratively training while simultaneously adapting to each client's local applications. Existing PFL methods typically face a significant challenge due to the ubiquitous data heterogeneity (i.e., non-IID data) across clients, which severely hinders convergence and degrades performance. We identify that the root issue lies in the long-standing reliance on gradient-based updates, which are inherently sensitive to non-IID data. To fundamentally address this issue and bridge the research gap, in this paper, we propose a Heterogeneity-invariant Personalized Federated learning scheme, named FedHiP, through analytical (i.e., closed-form) solutions to avoid gradient-based updates. Specifically, we exploit the trend of self-supervised pre-training, leveraging a foundation model as a frozen backbone for gradient-free feature extraction. Following the feature extractor, we further develop an analytic classifier for gradient-free training. To support both collective generalization and individual personalization, our FedHiP scheme incorporates three phases: analytic local training, analytic global aggregation, and analytic local personalization. The closed-form solutions of our FedHiP scheme enable its ideal property of heterogeneity invariance, meaning that each personalized model remains identical regardless of how non-IID the data are distributed across all other clients. Extensive experiments on benchmark datasets validate the superiority of our FedHiP scheme, outperforming the state-of-the-art baselines by at least 5.79%-20.97% in accuracy.

联邦学习个性化异构性闭式解

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。