提出高效分布式专家学习算法，通信量近最优且抗自适应攻击
Communication Bounds for the Distributed Experts Problem
- 设计通信高效的分布式专家算法，支持多种聚合方式
- 在HPO-B基准上实测通信量显著降低，性能接近理论最优
- 适用于大规模分布式优化场景，如联邦学习与超参调优
本文研究分布式环境下的专家问题，其中专家代价需在多个服务器间聚合。考虑消息传递与广播等通信模型，以及求和与ℓ_p范数等多种聚合函数。提出首个通信高效的协议，在强自适应对手下仍能达到近似最优的累积损失（regret）。进一步给出条件性下界，证明该通信量近乎最优。最后在HPO-B基准上实现协议原型，验证了实际通信开销的显著节省。
原文摘要 · Abstract (English)
In this work, we study the experts problem in the distributed setting where an expert's cost needs to be aggregated across multiple servers. Our study considers various communication models such as the message-passing model and the broadcast model, along with multiple aggregation functions, such as summing and taking the $\ell_p$ norm of an expert's cost across servers. We propose the first communication-efficient protocols that achieve near-optimal regret in these settings, even against a strong adversary who can choose the inputs adaptively. Additionally, we give a conditional lower bound showing that the communication of our protocols is nearly optimal. Finally, we implement our protocols and demonstrate empirical savings on the HPO-B benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。