arXiv:2503.20120stat.MLcs.LG2025-03被引 2

用柯西损失提升核岭回归抗噪能力,适用于重尾和异常值数据。

On the Robustness of Kernel Ridge Regression Using the Cauchy Loss Function

  • 引入广义柯西噪声框架,支持任意阶矩有限的噪声分布。
  • 证明柯西损失下模型风险与平方误差风险等价,实现近乎最优收敛率。
  • 实验验证在合成与真实数据上对多种噪声均具鲁棒性,适合高噪声场景使用。

稳健回归旨在处理存在异常值、重尾分布或污染数据时的回归函数估计问题,这类情况会严重影响模型性能。现有理论多假设噪声具有有限绝对均值,但该假设不适用于柯西分布、部分帕累托分布等。本文提出一种广义柯西噪声框架,涵盖所有具有有限任意阶矩的噪声分布,即使绝对均值无穷也适用。在此框架下,研究基于核函数的柯西岭回归器(KCRR),其通过最小化正则化的经验柯西风险实现鲁棒性。为推导KCRR的$L_2$-风险界,建立过量柯西风险与$L_2$-风险在柯西损失尺度参数足够大时的等价关系。进一步,在回归函数满足霍尔德光滑性条件下,推导出KCRR的过量柯西风险界,表明随着尺度参数减小性能提升。结合尺度参数对风险的双重影响及其与$L_2$-风险的等价性,建立了KCRR在$L_2$-风险下的几乎极小极大最优收敛率,凸显柯西损失在应对各类噪声时的稳健性。最后,通过在合成及真实数据集上的实验验证了KCRR在多种噪声污染场景下的有效性。

原文摘要 · Abstract (English)

Robust regression aims to develop methods for estimating an unknown regression function in the presence of outliers, heavy-tailed distributions, or contaminated data, which can severely impact performance. Most existing theoretical results in robust regression assume that the noise has a finite absolute mean, an assumption violated by certain distributions, such as Cauchy and some Pareto noise. In this paper, we introduce a generalized Cauchy noise framework that accommodates all noise distributions with finite moments of any order, even when the absolute mean is infinite. Within this framework, we study the \textit{kernel Cauchy ridge regressor} (\textit{KCRR}), which minimizes a regularized empirical Cauchy risk to achieve robustness. To derive the $L_2$-risk bound for KCRR, we establish a connection between the excess Cauchy risk and $L_2$-risk for sufficiently large scale parameters of the Cauchy loss, which reveals that these two risks are equivalent. Furthermore, under the assumption that the regression function satisfies Hölder smoothness, we derive excess Cauchy risk bounds for KCRR, showing improved performance as the scale parameter decreases. By considering the twofold effect of the scale parameter on the excess Cauchy risk and its equivalence with the $L_2$-risk, we establish the almost minimax-optimal convergence rate for KCRR in terms of $L_2$-risk, highlighting the robustness of the Cauchy loss in handling various types of noise. Finally, we validate the effectiveness of KCRR through experiments on both synthetic and real-world datasets under diverse noise corruption scenarios.

稳健回归核方法柯西损失抗噪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。