提出新型鲁棒平滑损失函数,提升模型在异常数据下的训练效果
Designing a Robust, Bounded, and Smooth Loss Function for Improved Supervised Learning
- 设计兼具鲁棒性、有界性和光滑性的新损失函数
- 在真实数据和注入异常数据上均显著优于基准模型
- 适合处理含噪声或异常值的时序预测任务
损失函数在机器学习中至关重要,尤其在监督学习框架下。传统损失函数在高维和对异常值敏感的数据集上表现不佳,常导致性能下降和训练收敛变慢。本文提出一种鲁棒、有界且光滑的损失函数(RoBoS-NN),并从理论上分析其泛化能力以严格证明其鲁棒性。将该损失函数融入神经网络框架,构建新的鲁棒算法 $\ ext{L}_{\ ext{RoBoS}}$-NN,用于时序预测。在多个真实世界数据集上进行实验,并注入异常值测试其在挑战性场景下的表现。数值结果表明,$\ ext{L}_{\ ext{RoBoS}}$-NN 在准确率指标上显著优于其他基准模型。
原文摘要 · Abstract (English)
The loss function is crucial to machine learning, especially in supervised learning frameworks. It is a fundamental component that controls the behavior and general efficacy of learning algorithms. However, despite their widespread use, traditional loss functions have significant drawbacks when dealing with high-dimensional and outlier-sensitive datasets, which frequently results in reduced performance and slower convergence during training. In this work, we develop a robust, bounded, and smooth (RoBoS-NN) loss function to resolve the aforementioned hindrances. The generalization ability of the loss function has also been theoretically analyzed to rigorously justify its robustness. Moreover, we implement RoboS-NN loss in the framework of a neural network (NN) to forecast time series and present a new robust algorithm named $\mathcal{L}_{\text{RoBoS}}$-NN. To assess the potential of $\mathcal{L}_{\text{RoBoS}}$-NN, we conduct experiments on multiple real-world datasets. In addition, we infuse outliers into data sets to evaluate the performance of $\mathcal{L}_{\text{RoBoS}}$-NN in more challenging scenarios. Numerical results show that $\mathcal{L}_{\text{RoBoS}}$-NN outperforms the other benchmark models in terms of accuracy measures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。