提出新框架提升联邦学习中少数客户端的公平性。
Federated Multi-Objective Learning with Controlled Pareto Frontiers
- 用偏好锥约束实现每个客户端的帕累托最优。
- 实验显示提升客户端公平性，训练足够久后精度可接近FedAvg。
- 适合关注隐私保护下多目标公平性的研究者。
联邦学习（FL）是一种广泛采用的隐私保护模型训练范式，但FedAvg倾向于优化多数客户端而忽视少数。现有方法如联邦多目标学习（FMOL）虽引入多目标优化（MOO），但仅能获得任务级帕累托驻点，客户端公平性仍取决于随机性。本文提出锥正则化联邦多目标学习（CR-FMOL），首个通过新颖偏好锥约束强制客户端级帕累托最优的联邦多目标优化框架。经过本地联邦多梯度下降平均（FMGDA）/联邦随机多梯度下降平均（FSMGDA）步骤后，各客户端将聚合的任务损失向量作为隐式偏好上传；服务器随后求解以均匀向量为中心的锥约束帕累托多任务学习子问题，生成对每个客户端在其锥内帕累托驻点的下降方向。在非独立同分布（non-IID）基准上的实验表明，CR-FMOL显著提升客户端公平性，尽管初期性能略低于FedAvg，但经充分训练后可达到相近准确率。
原文摘要 · Abstract (English)
Federated learning (FL) is a widely adopted paradigm for privacy-preserving model training, but FedAvg optimise for the majority while under-serving minority clients. Existing methods such as federated multi-objective learning (FMOL) attempts to import multi-objective optimisation (MOO) into FL. However, it merely delivers task-wise Pareto-stationary points, leaving client fairness to chance. In this paper, we introduce Conically-Regularised FMOL (CR-FMOL), the first federated MOO framework that enforces client-wise Pareto optimality through a novel preference-cone constraint. After local federated multi-gradient descent averaging (FMGDA) / federated stochastic multi-gradient descent averaging (FSMGDA) steps, each client transmits its aggregated task-loss vector as an implicit preference; the server then solves a cone-constrained Pareto-MTL sub-problem centred at the uniform vector, producing a descent direction that is Pareto-stationary for every client within its cone. Experiments on non-IID benchmarks show that CR-FMOL enhances client fairness, and although the early-stage performance is slightly inferior to FedAvg, it is expected to achieve comparable accuracy given sufficient training rounds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。