arXiv:2601.20571cs.LGcs.AI2026-01

提出轻量级异步算法AsylADMM,解决边缘设备非光滑鲁棒学习难题

Fast and Efficient Gossip Algorithms for Robust and Non-smooth Decentralized Learning

  • 设计仅需每节点两个变量的异步共识算法,内存占用极低
  • 在分位数、几何中位数等非光滑任务上收敛速度优于现有方法
  • 适合资源受限的边缘计算场景,尤其适用于含噪声数据的分布式学习

在资源受限的边缘设备上进行去中心化学习,需要通信高效、抗数据污染且内存轻量的算法。现有基于消息传递的方法虽能实现通信高效,但难以保证鲁棒性;而处理非光滑目标(如pinball损失、ℓ₁损失)的鲁棒优化方法通常依赖非光滑损失,但标准消息传递算法主要针对光滑损失设计。已有异步去中心化ADMM方法虽可处理非光滑问题,但其内存开销随节点度数增长,不适用于内存受限场景。本文提出AsylADMM——一种新型异步消息传递算法,用于去中心化非光滑优化,每个节点仅需维护两个变量。我们为同步版本提供了新理论分析,并据此在平方损失简化设置下证明了AsylADMM的收敛性。实验表明,AsylADMM在分位数估计、几何中位数、Lasso回归和鲁棒回归等挑战性非光滑问题上均比现有基线更快收敛。本工作为实现鲁棒且非光滑的去中心化学习开辟了可行路径。

原文摘要 · Abstract (English)

Decentralized learning on resource-constrained edge devices demands algorithms that are communication-efficient, robust to data corruption, and lightweight in memory. State-of-the-art gossip-based methods address communication efficiency, but achieving robustness remains challenging. Methods for robust estimation and optimization typically rely on non-smooth objectives (\textit{e.g.}, pinball loss, $\ell_1$ loss), yet standard gossip methods are primarily designed for smooth losses. Asynchronous decentralized ADMM-based methods have been proposed to handle such non-smooth objectives; however, existing approaches require memory that scales with node degree, making them impractical when memory is limited. We propose AsylADMM, a novel asynchronous gossip algorithm for decentralized non-smooth optimization requiring only two variables per node. We provide a new theoretical analysis for the synchronous variant and leverage it to prove convergence of AsylADMM in a simplified setting based on the squared loss. Empirically, AsylADMM converges faster than existing baselines on challenging non-smooth problems, including quantile and geometric median estimation, lasso regression, and robust regression. More broadly, our novel gossip framework opens a practical pathway toward robust and non-smooth decentralized learning.

去中心化学习非光滑优化异步算法边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。