提出无需梯度的黎曼流形联邦学习方法,降低计算成本。
Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach
- 用欧氏随机扰动替代切空间采样,简化零阶梯度估计
- 理论证明算法收敛速度与一阶方法相当,具亚线性收敛性
- 适用于无梯度或参数受限场景,如对抗攻击与低秩训练
联邦学习(FL)作为一种在保护数据隐私的前提下协同训练模型的范式已崭露头角。然而,现有算法主要针对无约束优化问题并依赖精确梯度信息,限制了其在仅可获取噪声函数值或参数受约束场景下的应用。为此,本文提出一种基于投影的零阶黎曼流形联邦学习新算法。通过引入投影算子,设计了一种计算高效的零阶黎曼梯度估计器。与现有方法不同,该估计器仅需简单的欧氏随机扰动,无需在切空间中采样随机向量,从而降低计算开销。理论上,我们首先证明了估计器的近似性质,并建立了所提算法的亚线性收敛性,收敛速率与一阶方法一致。数值实验中,我们首先通过核主成分分析评估估计器效率;进一步将该算法应用于两个真实场景:深度神经网络的零阶攻击和低秩神经网络训练,验证了理论结论。
原文摘要 · Abstract (English)
Federated learning (FL) has emerged as a powerful paradigm for collaborative model training across distributed clients while preserving data privacy. However, existing FL algorithms predominantly focus on unconstrained optimization problems with exact gradient information, limiting its applicability in scenarios where only noisy function evaluations are accessible or where model parameters are constrained. To address these challenges, we propose a novel zeroth-order projection-based algorithm on Riemannian manifolds for FL. By leveraging the projection operator, we introduce a computationally efficient zeroth-order Riemannian gradient estimator. Unlike existing estimators, ours requires only a simple Euclidean random perturbation, eliminating the need to sample random vectors in the tangent space, thus reducing computational cost. Theoretically, we first prove the approximation properties of the estimator and then establish the sublinear convergence of the proposed algorithm, matching the rate of its first-order counterpart. Numerically, we first assess the efficiency of our estimator using kernel principal component analysis. Furthermore, we apply the proposed algorithm to two real-world scenarios: zeroth-order attacks on deep neural networks and low-rank neural network training to validate the theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。