arXiv:2601.08216cs.LGcs.CR2026-01

一次通信即可完成联邦岭回归,精确恢复全局解。

One-Shot Federated Ridge Regression: Exact Recovery via Sufficient Statistic Aggregation

  • 客户端仅需发送局部统计量,服务器单次求逆即得全局解。
  • 在特征覆盖条件下可实现精确恢复,非近似解;否则有谱依赖误差界。
  • 通信量降至O(d²),支持差分隐私且无多轮隐私衰减问题。

联邦学习通常需多轮客户端与服务器通信,收敛速度受学习率、数据异质性和客户端采样影响。本文探讨分布式线性回归是否必须迭代通信。结果表明:无需迭代。将联邦岭回归建模为分布式均衡问题,各客户端计算本地充分统计量——格拉姆矩阵与矩向量——并仅传输一次。服务器通过单次矩阵求逆重构全局解。证明了精确恢复:在客户端特征矩阵满足覆盖条件时,一次聚合即可获得集中式岭回归解,而非近似解。对于不满足覆盖条件的异质分布,推导出依赖于聚合格拉姆矩阵谱性质的非渐近误差界。通信复杂度从迭代方法的$\mathcal{O}(Rd)$降低至$\mathcal{O}(d^2)$;在高维场景下,提出并验证随机投影技术将通信量进一步降至$\mathcal{O}(m^2)$($m \ll d$)。建立差分隐私保障,噪声每客户注入一次,避免多轮协议中的隐私累积衰减。此外考虑客户端掉线鲁棒性、联邦交叉验证调参及与梯度方法对比。综合实验在合成异质回归数据上显示,一次融合性能媲美FedAvg,通信量减少最高达38倍。该框架适用于核方法与随机特征模型,但不适用于一般非线性架构。

原文摘要 · Abstract (English)

Federated learning protocols require repeated synchronization between clients and a central server, with convergence rates depending on learning rates, data heterogeneity, and client sampling. This paper asks whether iterative communication is necessary for distributed linear regression. We show it is not. We formulate federated ridge regression as a distributed equilibrium problem where each client computes local sufficient statistics -- the Gram matrix and moment vector -- and transmits them once. The server reconstructs the global solution through a single matrix inversion. We prove exact recovery: under a coverage condition on client feature matrices, one-shot aggregation yields the centralized ridge solution, not an approximation. For heterogeneous distributions violating coverage, we derive non-asymptotic error bounds depending on spectral properties of the aggregated Gram matrix. Communication reduces from $\mathcal{O}(Rd)$ in iterative methods to $\mathcal{O}(d^2)$ total; for high-dimensional settings, we propose and experimentally validate random projection techniques reducing this to $\mathcal{O}(m^2)$ where $m \ll d$. We establish differential privacy guarantees where noise is injected once per client, eliminating the composition penalty that degrades privacy in multi-round protocols. We further address practical considerations including client dropout robustness, federated cross-validation for hyperparameter selection, and comparison with gradient-based alternatives. Comprehensive experiments on synthetic heterogeneous regression demonstrate that one-shot fusion matches FedAvg accuracy while requiring up to $38\times$ less communication. The framework applies to kernel methods and random feature models but not to general nonlinear architectures.

联邦学习岭回归一次通信差分隐私

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。