arXiv:2503.04370cs.LG2025-03

提出新评估指标与集成方法,解决分类模型在不平衡数据中的偏差问题。

FILM: Framework for Imbalanced Learning Machines based on a new unbiased performance measure and a new ensemble-based technique

  • 设计无偏整合系数UIC,降低少数类评估偏差(p<10⁻⁴)
  • 引入IPIP集成算法,在7个数据集中有3个优于基准方法
  • 适用于需要可靠模型选择的不平衡二分类场景

本研究针对二分类任务中不平衡数据集的挑战。在这些情况下,标准评估指标常因少数类样本比例过低而产生偏差。我们在七个数据集上实验发现,现有指标在判断哪个模型表现更优时存在不一致,这说明需要一种更一致、无偏的评估指标以支持稳健的模型选择。为此,我们提出一种新指标——无偏整合系数(UIC),通过聚合现有指标并惩罚易受不平衡影响的度量,显著降低对少数类的偏差(p < 10⁻⁴)。此外,我们引入基于集成的IPIP算法,用于处理不平衡学习问题。实验表明,在使用随机森林和逻辑回归模型时,IPIP在7个数据集中的3个上优于其他已知的平衡感知方法,验证了其有效性。该框架已实现为FILM(不平衡学习机器框架)R包,可在https://github.com/antoniogt/FILM获取。

原文摘要 · Abstract (English)

This research addresses the challenges of handling unbalanced datasets for binary classification tasks. In such scenarios, standard evaluation metrics are often biased by the disproportionate representation of the minority class. Conducting experiments across seven datasets, we uncovered inconsistencies in evaluation metrics when determining the model that outperforms others for each binary classification problem. This justifies the need for a metric that provides a more consistent and unbiased evaluation across unbalanced datasets, thereby supporting robust model selection. To mitigate this problem, we propose a novel metric, the Unbiased Integration Coefficients (UIC), which exhibits significantly reduced bias ($p < 10^{-4}$) towards the minority class compared to conventional metrics. The UIC is constructed by aggregating existing metrics while penalising those more prone to imbalance. In addition, we introduce the Identical Partitions for Imbalance Problems (IPIP) algorithm for imbalanced ML problems, an ensemble-based approach. Our experimental results show that IPIP outperforms other baseline imbalance-aware approaches using Random Forest and Logistic Regression models in three out of seven datasets as assessed by the UIC metric, demonstrating its effectiveness in addressing imbalanced data challenges in binary classification tasks. This new framework for dealing with imbalanced datasets is materialized in the FILM (Framework for Imbalanced Learning Machines) R Package, accessible at https://github.com/antoniogt/FILM.

不平衡学习评估指标集成方法

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。