提出一类关注相对误差的损失函数,适合比例关系明显的回归任务。
Ratio-based Loss Functions
- 基于输出与预测值的比值设计损失函数,强调相对误差而非绝对差。
- 系统分析了此类损失函数的连续性、凸性和可微性等通用性质。
- 为未来研究比例型回归模型提供理论基础,适合数据有尺度依赖性的场景。
机器学习算法依赖三个关键要素:风险函数(损失函数的期望)、函数空间(假设空间)和允许的概率测度集。本文综述一类称为比率基的损失函数。监督学习中,分类常用基于边距的损失函数(依赖于标签值 $y_i$ 与预测值 $f(x_i)$ 的乘积),回归则常见基于距离的损失函数(依赖于 $y_i$ 与 $f(x_i)$ 的差值),适用于信号加噪声的加法模型假设。然而,文献中部分回归损失函数针对乘法误差结构,关注相对误差,即 $y_i$ 与 $f(x_i)$ 的比值。本文系统研究这类比率基损失函数,提出若干新损失函数,供未来研究参考。重点分析其连续性、Lipschitz连续性、凸性和可微性等通用性质,这些性质在多数机器学习算法中至关重要。本文不聚焦特定算法推导一致性、学习率或稳定性结果,旨在为该方向的后续研究奠定基础。
原文摘要 · Abstract (English)
Algorithms in machine learning and AI do critically depend on at least three key components: (i) the risk function, which is the expectation of the loss function, (ii) the function space, which is often called the hypothesis space, and (iii) the set of probability measures, which are allowed for the specified algorithm. This paper gives a survey of a certain class of loss functions, which we call ratio-based. In supervised learning, margin-based loss functions for classification tasks depending on the product of the output values $y_i$ and the predictions $f(x_i)$ as well as distance-based loss functions depending on the difference of $y_i$ and $f(x_i)$ for regression are common. Distance-based loss functions are in particular useful, if an additive model assumption seems plausible, i.e. the common signal plus noise assumption. However, in the literature, several loss functions proposed for regression purposes have a multiplicative error structure in mind and pay attention to relative errors, i.e. to the ratio of $y_i$ and $f(x_i)$. In this survey article, we systematically investigate such ratio-based loss functions and propose a few new losses, which may be interesting for future research. We concentrate on investigating general properties of ratio-based loss functions like continuity, Lipschitz-continuity, convexity, and differentiability, because these properties play a central role in most machine learning algorithms. Therefore, we do not focus on some specific machine learning algorithm to derive universal consistency, learning rates, or stability results. Instead, we want to enable future research in this direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。