arXiv:2602.14440stat.MEcs.LG2026-02

将排序与尺度解耦,提升回归对异常值的鲁棒性

CAIRO: Decoupling Order from Scale in Regression

  • 先用排名损失学习排序,再通过保序回归恢复真实尺度
  • 在重尾和异方差噪声下性能显著优于传统回归方法
  • 适合对噪声敏感或需可靠预测置信度的场景

标准回归方法通常优化单一点对点目标(如均方误差),将排序学习与尺度学习耦合,使模型易受异常值和重尾噪声影响。我们提出CAIRO(Calibrate After Initial Rank Ordering)框架,将回归分为两个阶段:第一阶段通过最小化尺度不变的排名损失学习评分函数;第二阶段利用保序回归恢复目标尺度。我们理论上刻画了一类“最优排序”目标函数(包括RankNet和Gini协方差变体),证明其在弱假设下可恢复真实条件均值的排序。进一步证明后续单调校准可在总体层面恢复真实回归函数,并数学保证有限样本预测严格自校准。实验表明,CAIRO结合神经网络表征学习与基于排名的统计稳健性,在表格基准上达到主流树集成模型性能,并在重尾或异方差噪声环境下显著超越标准回归目标。

原文摘要 · Abstract (English)

Standard regression methods typically optimize a single pointwise objective, such as mean squared error, which conflates the learning of ordering with the learning of scale. This coupling renders models vulnerable to outliers and heavy-tailed noise. We propose CAIRO (Calibrate After Initial Rank Ordering), a framework that decouples regression into two distinct stages. In the first stage, we learn a scoring function by minimizing a scale-invariant ranking loss; in the second, we recover the target scale via isotonic regression. We theoretically characterize a class of "Optimal-in-Rank-Order" objectives -- including variants of RankNet and Gini covariance -- and prove that they recover the ordering of the true conditional mean under mild assumptions. We further show that subsequent monotone calibration recovers the true regression function at the population level and mathematically guarantees that finite-sample predictions are strictly auto-calibrated. Empirically, CAIRO combines the representation learning of neural networks with the robustness of rank-based statistics. It matches the performance of state-of-the-art tree ensembles on tabular benchmarks and significantly outperforms standard regression objectives in regimes with heavy-tailed or heteroskedastic noise.

回归鲁棒学习排序校准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。