提出更严格的单峰正则化方法,提升小样本序数回归预测精度。
Unimodality-Promoting Regularized Learning for Ordinal Regression
- 设计新正则项,严格约束预测概率分布呈单峰形态。
- 在小样本下显著降低预测方差,提升准确率与稳定性。
- 适用于医疗评分、满意度评估等有序标签场景。
序数回归是针对具有自然顺序关系的分类变量的建模任务。以往研究表明,真实世界中的序数数据在解释变量取值较大范围内,其条件概率分布(CPD)通常呈现单峰特性,即使在剩余区域也接近单峰。因此,促进预测CPD向单峰逼近的正则化学习方法(UPRL)能有效降低预测方差,尤其在小样本情况下表现优异。然而本文发现,现有UPRL方法不仅使分布趋近单峰,还导致其整体尺度增大(即更平滑、置信度更低),引入了额外偏差。为此,我们提出一种新方法,更严格地实现单峰性并避免尺度相关偏差。实验表明,该方法在小样本或大样本训练下均优于先前方法;分析揭示其性能优势源于消除了意外的尺度偏差。
原文摘要 · Abstract (English)
Ordinal regression, also called ordinal classification, is classification of ordinal data, in which the underlying target variable is categorical and considered to have a natural ordinal relation. Previous works have indicated that, in many real-world ordinal data, the conditional probability distribution (CPD) of the target variable given a value of the explanatory variable would be unimodal in a large domain of the explanatory variable and close to be unimodal even in a remaining domain. Therefore, unimodality-promoting regularized learning (UPRL), which promotes a predicted CPD closer to be unimodal with the aim of decreasing a prediction variance without inducing much bias for ordinal data of the unimodality, is promising to improve the prediction performance especially with small-size training data. In this study, we show that previous UPRL methods promote a predicted CPD to not only become closer to be unimodal but also have a larger scale (in other words, be smoother or less-confident). Therefore, we develop a novel method that more strictly reflects the idea of UPRL and evades a scale-related bias, and verify through experimental comparison that the unimodality-promotion indeed contributes to improve the prediction performance. Additionally, while our proposed UPRL method could perform better for smaller-scale data or with larger-size training data compared to a previous UPRL method, our analysis explains this experimental observation in terms of the presence or absence of an unexpected scale-related bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。