提出DEPOSITUM算法，解决去中心化联邦学习中的非凸优化难题。
Decentralized Nonconvex Composite Federated Learning with Gradient Tracking and Momentum
- 用梯度追踪机制保持动量滤波的随机梯度估计，支持多种动量策略。
- 在无界梯度异质性假设下，达到O(1/ε²)的迭代复杂度，收敛性更强。
- 适合数据异构、通信成本高的实际联邦学习场景，实验验证效果优。
去中心化联邦学习（DFL）无需中央服务器即可协同训练模型。当局部目标为非凸且与弱凸正则项耦合时，DFL面临包含数据异构性、随机梯度噪声和一致性误差的复杂优化问题。本文提出DEPOSITUM算法，通过追踪机制维护动量滤波的随机梯度估计，兼容Polyak和Nesterov动量。该算法允许通信轮次间进行多次本地更新，提升通信效率。理论分析表明，其在不假设有界梯度异质性或均方光滑性条件下，达到期望ε-平稳点的迭代复杂度为 cal O(1/ε^2)。采用合适步长与动量调度后，平均平稳性指标在经历网络依赖的瞬态期后，可达到 cal O(1/ qrt{nT})的速率，且小批量大小与总轮数T无关。在多个基准数据集上的实验验证了该方法的有效性，并表现出与代表性联邦复合优化方法相当的竞争力。
原文摘要 · Abstract (English)
Decentralized Federated Learning (DFL) enables collaborative model training without relying on a central server. When local objectives are nonconvex and coupled with nonsmooth weakly convex regularization, DFL gives rise to a challenging decentralized nonconvex composite optimization problem involving data heterogeneity, stochastic-gradient noise, and consensus error. We propose DEPOSITUM, a decentralized composite optimization algorithm for this problem. DEPOSITUM maintains momentum-filtered stochastic gradient estimates via a tracking mechanism, which accommodates both Polyak and Nesterov momentum. It further allows multiple local updates between communication rounds to improve communication efficiency. Theoretical analysis demonstrates that it achieves an expected $ε$-stationary point with an iteration complexity of $\mathcal{O}(1/ε^2)$ without imposing bounded gradient heterogeneity or mean-squared smoothness assumptions. With an appropriate stepsize and momentum schedule, the averaged stationarity measure further achieves a rate of \(\mathcal{O}(1/\sqrt{nT})\) after a network-dependent transient, using a mini-batch size independent of $T$. Experiments on different benchmark datasets validate the effectiveness of DEPOSITUM and demonstrate competitive performance against representative federated composite optimization methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。