提出压缩通信下的去中心化非光滑非凸优化统一框架，提升通信效率。
Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimization
- 统一多种带压缩的去中心化随机次梯度方法，支持无偏与有偏压缩
- 证明在非光滑非凸条件下全局收敛，无需 Clarke 正则性假设
- 设计新型压缩方法，适合通信受限场景的分布式学习
本文研究具有压缩通信的去中心化非光滑非凸优化问题。提出一个通用框架，统一了多种带无偏压缩和带误差补偿的压缩方法。通过将共识误差迭代与平均迭代关联到连续时间微分包含的轨迹，证明了该框架下所有方法在目标函数非光滑且缺乏 Clarke 正则性时的全局收敛性。基于此框架，进一步设计了若干基于压缩的方法，包括采用符号正则化和梯度追踪动量的去中心化随机次梯度法。初步数值实验验证了理论结果，并揭示了新方法在通信效率与精度之间的权衡。
原文摘要 · Abstract (English)
In this paper, we consider the nonsmooth nonconvex decentralized optimization problem, where inter-agent communication is compressed. We propose a general framework that unifies various decentralized stochastic subgradient-type methods with unbiased compression and contractive compression with error compensation. By relating the consensus-error iterates and the averaged iterates to the trajectories of continuous-time differential inclusions, we establish global convergence for all methods encompassed by our framework when the objective functions are nonsmooth and lack Clarke regularity. Based on our framework, we further develop several compression-based methods, including decentralized stochastic subgradient methods utilizing sign-based regularization and gradient-tracking momentum. Preliminary numerical experiments empirically support our theoretical results and highlight the communication-accuracy trade-off of the newly developed methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。