通过双层优化实现高效客户端协作，提升异构数据下的模型性能。
CoBo: Collaborative Learning via Bilevel Optimization
- 将客户端选择与模型训练建模为双层优化问题，协同求解。
- 在80个异构客户端上，准确率比主流个性化算法高9.3%。
- 算法可扩展且弹性强，适合大规模分布式学习场景。
协作学习通过客户端间通信提升多客户端训练效率，但有效客户端的识别极具挑战性，常伴随显著开销。本文将客户端选择与模型训练建模为两个相互关联的优化问题，提出一种新型的双层优化框架用于协作学习。我们引入CoBo，一种基于SGD的交替优化算法，具有可扩展性和弹性，能高效求解该问题并提供理论收敛保证。实验表明，CoBo在高异构性任务中表现优异，在80个客户端分布的数据集上，准确率较主流个性化算法提升9.3%。
原文摘要 · Abstract (English)
Collaborative learning is an important tool to train multiple clients more effectively by enabling communication among clients. Identifying helpful clients, however, presents challenging and often introduces significant overhead. In this paper, we model client-selection and model-training as two interconnected optimization problems, proposing a novel bilevel optimization problem for collaborative learning. We introduce CoBo, a scalable and elastic, SGD-type alternating optimization algorithm that efficiently addresses these problem with theoretical convergence guarantees. Empirically, CoBo achieves superior performance, surpassing popular personalization algorithms by 9.3% in accuracy on a task with high heterogeneity, involving datasets distributed among 80 clients.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。