arXiv:2603.11992cs.AIcs.LG2026-03

用少量共享模型实现大规模个性化联邦学习,提升效率与效果。

Few-for-Many Personalized Federated Learning

  • 仅维护K个服务器模型服务所有客户,替代传统需M个模型的方案。
  • 实验表明3个模型即超越现有顶尖方法,在视觉、NLP与医疗图像上均有效。
  • 无需人工分组或调参,自动发现最优模型多样性,适合大规模部署。

个性化联邦学习(PFL)旨在为数据分布高度异构的客户端训练定制化模型,同时保护数据隐私。现有方法多依赖聚类或模型插值等启发式策略,缺乏对异构客户端目标间权衡的理论支撑。服务M个具有不同数据分布的客户端本质上是多目标优化问题,理想情况下需在帕累托前沿保持M个独立模型。然而在包含数百甚至上千客户端的联邦场景中,维护M个模型存在显著可扩展性挑战。为此,本文将PFL重构为‘少对多’优化问题,仅维护K个共享服务器模型(K≪M),以共同服务全部M个客户端。理论证明该框架可实现近似最优个性化:随着K增大,近似误差减小;当数据量增长时,每个客户端模型趋近其最优解。基于此,提出实用算法FedFew,通过高效的梯度更新联合优化这K个服务器模型。与依赖手动客户划分的聚类方法或需精细调参的插值方法不同,FedFew通过优化过程自动发现最优模型多样性。在视觉、自然语言处理及真实世界医学影像数据集上的实验表明,仅使用3个模型的FedFew始终优于其他最先进方法。代码已公开于https://github.com/pgg3/FedFew。

原文摘要 · Abstract (English)

Personalized Federated Learning (PFL) aims to train customized models for clients with highly heterogeneous data distributions while preserving data privacy. Existing approaches often rely on heuristics like clustering or model interpolation, which lack principled mechanisms for balancing heterogeneous client objectives. Serving $M$ clients with distinct data distributions is inherently a multi-objective optimization problem, where achieving optimal personalization ideally requires $M$ distinct models on the Pareto front. However, maintaining $M$ separate models poses significant scalability challenges in federated settings with hundreds or thousands of clients. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only $K$ shared server models ($K \ll M$) to collectively serve all $M$ clients. We prove that this framework achieves near-optimal personalization: the approximation error diminishes as $K$ increases and each client's model converges to each client's optimum as data grows. Building on this reformulation, we propose FedFew, a practical algorithm that jointly optimizes the $K$ server models through efficient gradient-based updates. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew automatically discovers the optimal model diversity through its optimization process. Experiments across vision, NLP, and real-world medical imaging datasets demonstrate that FedFew, with just 3 models, consistently outperforms other state-of-the-art approaches. Code is available at https://github.com/pgg3/FedFew.

联邦学习个性化优化少对多

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。