arXiv:2506.05215cs.LG2025-06

提出一种去中心化鲁棒核学习算法,统一处理多种鲁棒回归任务。

Theory of Decentralized Robust Kernel-Based Learning

  • 基于核希尔伯特空间与网络图结构,设计去中心化鲁棒学习框架。
  • 在合理参数下达到最优学习速率(含对数因子),收敛性有严格证明。
  • 适用于需要鲁棒性与分布式计算的场景,如边缘智能与传感器网络。

我们提出一种基于再生核希尔伯特空间(RKHS)的去中心化鲁棒核学习算法,利用可表示为连通图的网络系统实现。通过窗口函数 $W$ 和鲁棒性缩放参数 $σ>0$ 构造的鲁棒损失函数 $\huaL_σ$ 能涵盖广泛类型的鲁棒损失。该算法提供统一的去中心化学习框架,区别于现有基于分治策略的分布式鲁棒核学习方法。我们严格建立学习理论并给出完整收敛分析:每个本地鲁棒估计器均可逼近回归函数。基于核积分算子技术,我们推导出局部逼近序列在均方距离、RKHS范数和泛化误差上的高置信度收敛界。此外,我们给出本地样本量的严谨选择规则,并证明在适当选择步长与参数 $σ$ 下,算法可在两种范数下实现最优学习速率(至对数因子)。参数 $σ$ 对提升鲁棒性和保证良好收敛性至关重要,算法中去中心化、样本选择、鲁棒性与收敛性的内在关联得以清晰体现。

原文摘要 · Abstract (English)

We propose a new decentralized robust kernel-based learning algorithm within the framework of reproducing kernel Hilbert spaces (RKHSs) by utilizing a networked system that can be represented as a connected graph. The robust loss function $\huaL_σ$ induced by a windowing function $W$ and a robustness scaling parameter $σ>0$ can encompass a broad spectrum of robust losses. Consequently, the proposed algorithm effectively provides a unified decentralized learning framework for robust regression, which fundamentally differs from the existing distributed robust kernel-based learning schemes, all of which are divide-and-conquer based. We rigorously establish a learning theory and offer comprehensive convergence analysis for the algorithm. We show each local robust estimator generated from the decentralized algorithm can be utilized to approximate the regression function. Based on kernel-based integral operator techniques, we derive general high confidence convergence bounds for the local approximating sequence in terms of the mean square distance, RKHS norm, and generalization error, respectively. Moreover, we provide rigorous selection rules for local sample size and show that, under properly selected step size and scaling parameter $σ$, the decentralized robust algorithm can achieve optimal learning rates (up to logarithmic factors) in both norms. The parameter $σ$ is shown to be essential for enhancing robustness and ensuring favorable convergence behavior. The intrinsic connection among decentralization, sample selection, robustness of the algorithm, and its convergence is clearly reflected.

去中心化鲁棒学习核方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。