arXiv:2602.14244stat.MLcs.LG2026-02

通过渐进式个性化集成学习,解决联邦学习中模型过拟合与偏差的权衡问题。

Federated Ensemble Learning with Progressive Model Personalization

  • 构建多阶段集成模型,逐步增加个性化组件深度并控制复杂度
  • 在多个数据集上显著降低客户端偏差,性能优于现有方法
  • 适合处理异构数据分布下的隐私保护分布式学习场景

联邦学习在分布式学习中保障隐私,但面临客户端间统计异质性问题。个性化联邦学习(PFL)通过客户端特定模型缓解此问题,常用方法将神经网络分解为共享特征提取器和客户端专属头部。然而,深层共享组件阻碍个性化,大尺寸本地头部在数据有限时易过拟合。现有方法依赖固定浅层头部,无法有效平衡该权衡。本文提出一种受提升启发的框架,为每个客户端构建T个模型组成的集成。在提升迭代中,逐步增加个性化组件深度,并通过低秩分解或宽度压缩系统控制其有效复杂度。该设计同时抑制过拟合并显著减少客户端偏差,允许更丰富的个性化表达。我们提供了理论分析,建立了依赖平均局部样本量和总客户端数的良好泛化界。特别地,证明了共享层复杂度被有效抑制,且提升轮数T的依赖通过参数缩减得到控制。此外,首次为解耦式PFL模型提供非线性泛化保证。在基准和真实数据集(如EMNIST、CIFAR-10/100、Sent140)上的大量实验表明,所提框架在异构数据分布下始终优于最先进PFL方法。

原文摘要 · Abstract (English)

Federated Learning provides a privacy-preserving paradigm for distributed learning, but suffers from statistical heterogeneity across clients. Personalized Federated Learning (PFL) mitigates this issue by considering client-specific models. A widely adopted approach in PFL decomposes neural networks into a shared feature extractor and client-specific heads. While effective, this design induces a fundamental tradeoff: deep or expressive shared components hinder personalization, whereas large local heads exacerbate overfitting under limited per-client data. Most existing methods rely on rigid, shallow heads, and therefore fail to navigate this tradeoff in a principled manner. In this work, we propose a boosting-inspired framework that enables a smooth control of this tradeoff. Instead of training a single personalized model, we construct an ensemble of $T$ models for each client. Across boosting iterations, the depth of the personalized component are progressively increased, while its effective complexity is systematically controlled via low-rank factorization or width shrinkage. This design simultaneously limits overfitting and substantially reduces per-client bias by allowing increasingly expressive personalization. We provide theoretical analysis that establishes generalization bounds with favorable dependence on the average local sample size and the total number of clients. Specifically, we prove that the complexity of the shared layers is effectively suppressed, while the dependence on the boosting horizon $T$ is controlled through parameter reduction. Notably, we provide a novel nonlinear generalization guarantee for decoupled PFL models. Extensive experiments on benchmark and real-world datasets (e.g., EMNIST, CIFAR-10/100, and Sent140) demonstrate that the proposed framework consistently outperforms state-of-the-art PFL methods under heterogeneous data distributions.

联邦学习个性化集成学习模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。