用卷积平滑方法实现高效分布式支持向量机学习
Efficient Distributed Learning over Decentralized Networks with Convoluted Support Vector Machine
- 用卷积技术平滑非光滑损失函数，提升优化效率
- 算法实现线性收敛，且能准确恢复真实参数支持集
- 适合高维数据分布式分类任务，尤其关注收敛速度的场景
本文研究去中心化网络中高维数据的高效分类问题。惩罚型支持向量机（SVM）广泛用于高维分类，但目标函数的双重非光滑性导致现有分布式学习方法收敛缓慢，多为亚线性速率。为此，本文提出一种基于卷积的平滑技术，对非光滑铰链损失进行处理，使损失函数保持凸性且光滑。在此基础上，设计了一种高效的广义交替方向乘子法（ADMM）算法，用于求解去中心化网络上的惩罚型SVM。理论贡献有二：一是证明该算法具有可证明的线性收敛性，且实现简单；二是经过足够多的迭代后，最终稀疏估计器达到近似最优统计收敛率，并能准确恢复真实参数的支持集。大量模拟与真实数据集实验验证了理论结果的有效性。
原文摘要 · Abstract (English)
This paper addresses the problem of efficiently classifying high-dimensional data over decentralized networks. Penalized support vector machines (SVMs) are widely used for high-dimensional classification tasks. However, the double nonsmoothness of the objective function poses significant challenges in developing efficient decentralized learning methods. Many existing procedures suffer from slow, sublinear convergence rates. To overcome this limitation, we consider a convolution-based smoothing technique for the nonsmooth hinge loss function. The resulting loss function remains convex and smooth. We then develop an efficient generalized alternating direction method of multipliers (ADMM) algorithm for solving penalized SVM over decentralized networks. Our theoretical contributions are twofold. First, we establish that our generalized ADMM algorithm achieves provable linear convergence with a simple implementation. Second, after a sufficient number of ADMM iterations, the final sparse estimator attains near-optimal statistical convergence and accurately recovers the true support of the underlying parameters. Extensive numerical experiments on both simulated and real-world datasets validate our theoretical findings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。