用贝叶斯视角拓展联邦ADMM，提升异构场景下模型性能。
Federated ADMM from Bayesian Duality
- 基于变分贝叶斯构建新框架，关联对偶结构。
- 在非独立同分布数据上实现最高7%准确率提升。
- 适合研究联邦学习与优化算法的学者参考。
我们提出一种新的贝叶斯方法来推广联邦交替方向乘子法（Federated ADMM）。研究表明，变分贝叶斯（VB）目标函数的解与对偶结构相关联，不仅与ADMM固定点结构相似，还可对其进行泛化。例如，在各向同性高斯族上优化时可恢复典型的ADMM更新；而对于其他指数族分布，可导出新扩展。这些扩展包括一种牛顿类变体，可在二次目标上一步收敛；以及一种类似Adam的变体，在深度异构场景中实现最高7%的准确率提升。本工作为推广ADMM及其他原始-对偶方法开辟了新的贝叶斯路径。
原文摘要 · Abstract (English)
We propose a new Bayesian approach to generalize the federated Alternating Direction Method of Multipliers (ADMM). We show that the solutions of variational-Bayesian (VB) objectives are associated with a duality structure that not only resembles the structure of ADMM's fixed-points but also generalizes it. For example, ADMM-like updates are recovered when the VB objective is optimized over the isotropic-Gaussian family, and new non-trivial extensions are obtained for other exponential-family distributions. These extensions include a Newton-like variant that converges in one step on quadratic objectives and an Adam-like variant that yields up to 7% accuracy boosts for deep heterogeneous cases. Our work opens a new Bayesian way to generalize ADMM and other primal-dual methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。