不使用重平衡技术,评估二分类器在类别不平衡下的真实表现。
Beyond Rebalancing: Benchmarking Binary Classifiers Under Class Imbalance Without Rebalancing Techniques
- 在真实与合成数据上测试多种分类器,不加任何重平衡处理。
- 少数类样本越少、数据越复杂,模型性能越差,但TabPFN和集成模型更稳定。
- 适合关注模型原生鲁棒性的研究者,尤其医疗与异常检测场景。
类别不平衡是监督分类中的重大挑战,尤其在医学诊断和异常检测等关键领域,少数类样本稀少。尽管已有大量研究探索重平衡技术,但对不使用此类技术时二分类器性能的评估仍不足。本研究旨在评估分类器‘原生状态’下的表现,不进行任何显式重平衡。我们系统性地评估了多种二分类器在真实与合成数据集上的鲁棒性,涵盖从少量到极少数类样本的渐进式减少情形,并以一次学习和少样本学习为基线。通过合成决策边界生成,模拟不同数据复杂度。除标准分类器外,还测试了欠采样、过采样及一类分类(OCC)方法在严重不平衡下的行为。结果表明,随着数据复杂度上升和少数类规模减小,分类难度显著增加。传统模型在极端不平衡下性能下降明显,而基于TabPFN和提升集成的先进模型保持更高性能与更好泛化能力。可视化与评估指标进一步验证了结论。本工作为不平衡学习中的模型选择提供指导,揭示无需依赖重平衡技术的分类器鲁棒性。
原文摘要 · Abstract (English)
Class imbalance poses a significant challenge to supervised classification, particularly in critical domains like medical diagnostics and anomaly detection where minority class instances are rare. While numerous studies have explored rebalancing techniques to address this issue, less attention has been given to evaluating the performance of binary classifiers under imbalance when no such techniques are applied. Therefore, the goal of this study is to assess the performance of binary classifiers "as-is", without performing any explicit rebalancing. Specifically, we systematically evaluate the robustness of a diverse set of binary classifiers across both real-world and synthetic datasets, under progressively reduced minority class sizes, using one-shot and few-shot scenarios as baselines. Our approach also explores varying data complexities through synthetic decision boundary generation to simulate real-world conditions. In addition to standard classifiers, we include experiments using undersampling, oversampling strategies, and one-class classification (OCC) methods to examine their behavior under severe imbalance. The results confirm that classification becomes more difficult as data complexity increases and the minority class size decreases. While traditional classifiers deteriorate under extreme imbalance, advanced models like TabPFN and boosting-based ensembles retain relatively higher performance and better generalization compared to traditional classifiers. Visual interpretability and evaluation metrics further validate these findings. Our work offers valuable guidance on model selection for imbalanced learning, providing insights into classifier robustness without dependence on explicit rebalancing techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。