三种方法应对分类不平衡，阈值校准最稳。
Balancing the Scales: A Comprehensive Study on Tackling Class Imbalance in Binary Classification
- 对比三种主流处理不平衡数据的方法
- 阈值校准在9000次实验中表现最稳定
- 适合需要高精度识别少数类的场景
二分类任务中的类别不平衡仍是机器学习的重大挑战，常导致少数类性能下降。本研究系统评估了三种常用策略：合成少数类过采样技术（SMOTE）、类别权重调优和决策阈值校准。在15种不同机器学习模型和30个跨领域数据集上进行总计9000次实验，以F1-score为主要评估指标，同时跟踪了包括F2-score、精确率、召回率、Brier-score、PR-AUC和AUC在内的9项附加指标。结果表明，三种策略均优于无干预基线，其中决策阈值校准表现出最强的一致性效果。然而，各数据集最优方法差异显著，凸显针对具体问题测试多种策略的重要性。研究为处理不平衡数据的实践者提供重要参考，并强调需开展数据集特定分析。
原文摘要 · Abstract (English)
Class imbalance in binary classification tasks remains a significant challenge in machine learning, often resulting in poor performance on minority classes. This study comprehensively evaluates three widely-used strategies for handling class imbalance: Synthetic Minority Over-sampling Technique (SMOTE), Class Weights tuning, and Decision Threshold Calibration. We compare these methods against a baseline scenario of no-intervention across 15 diverse machine learning models and 30 datasets from various domains, conducting a total of 9,000 experiments. Performance was primarily assessed using the F1-score, although our study also tracked results on additional 9 metrics including F2-score, precision, recall, Brier-score, PR-AUC, and AUC. Our results indicate that all three strategies generally outperform the baseline, with Decision Threshold Calibration emerging as the most consistently effective technique. However, we observed substantial variability in the best-performing method across datasets, highlighting the importance of testing multiple approaches for specific problems. This study provides valuable insights for practitioners dealing with imbalanced datasets and emphasizes the need for dataset-specific analysis in evaluating class imbalance handling techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。