纠正数据不平衡会损害概率校准，尤其下采样更严重，事后校准可有效修复。
The Hidden Cost of Resampling: How Imbalance Correction Degrades Probability Calibration in Tree Ensembles
- 对比了SMOTE与下采样对概率校准的影响，发现后者危害更大。
- 在不平衡比1.9~70的5个数据集上，校准误差增加0.009，但判别性能提升更显著。
- 一次后处理校准即可消除损伤，适合依赖概率决策的场景使用。
SMOTE与随机上下采样是解决类别不平衡的常用方法，通常仅以少数类准确率或F1值评估。已有研究指出下采样会因扭曲训练先验而损害概率校准。本文将此视角扩展至合成过采样（SMOTE），在五个公开数据集（不平衡比1.9–70）和两种集成模型（随机森林、梯度提升）上，通过十次随机种子与配对统计分析发现：(1) SMOTE确实带来轻微校准损失（ECE +0.009；Cliff's delta = +0.27，小到中等），但在所研究不平衡范围内其判别能力提升通常超过校准代价；(2) 随机下采样才是主要风险——其损害随不平衡程度急剧上升，在不平衡比为70的数据集上，ECE从0.008飙升至0.395，主要因训练集过小难以可靠估计概率；(3) 一次后处理校准（Platt或等距校准）可消除该损伤，使ECE降低最多66%，且对排序性能影响极小（AUC下降0.002，Cliff's delta = -0.07）；(4) 修复下采样问题的解析先验修正法不适用于SMOTE，因其不仅改变先验还扭曲类条件密度，故仍需依赖数据驱动的校准。建议不平衡学习研究同时报告校准与判别性能，实践中若决策依赖预测概率，应在重采样后进行校准。
原文摘要 · Abstract (English)
Resampling methods such as SMOTE and random under/over-sampling are standard tools for class-imbalanced classification, almost always evaluated by minority-class accuracy or F1. Prior work has established that undersampling degrades probability calibration by distorting the training prior [1]. We extend this lens to synthetic oversampling (SMOTE) and provide a practical, evidence-based guide to when calibration damage matters and how to fix it. Across five public datasets (imbalance ratio 1.9-70) and two ensemble models (random forest, gradient boosting), with ten seeds and paired statistics, we find: (1) SMOTE's calibration cost is real but small (ECE +0.009; Cliff's delta = +0.27, small-to-moderate) across the studied imbalance range (IR 1.9-70) and its discrimination gains typically outweigh the calibration penalty; (2) random undersampling is the genuine danger -- its damage grows sharply with imbalance, inflating ECE from 0.008 to 0.395 on a dataset with ratio 70, largely because the resulting training sets are too small to estimate probabilities reliably; (3) a single post-hoc recalibration step (Platt or isotonic) eliminates the damage, reducing ECE by up to 66% at a negligible ranking-power cost (AUC -0.002, Cliff's delta = -0.07); and (4) the analytic prior-shift correction that repairs undersampling does not transfer to SMOTE, because SMOTE distorts the class-conditional density rather than only the prior -- so data-driven recalibration remains necessary. We recommend that imbalanced-learning studies report calibration alongside discrimination, and that practitioners recalibrate after resampling whenever predicted probabilities drive decisions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。