提出新评估指标LxCIM,解决分类模型在局部类别交换下的性能评价难题。
LxCIM: a new rank-based binary classifier performance metric invariant to local exchange of classes
- 基于排序的新型指标,对局部类别交换保持不变性。
- 与AUROC、准确率等有理论关联,可提供更细致的分析视角。
- 适合因果发现等具有类别交换不变性的任务,提升评估可信度。
二分类是机器学习中最古老、最普遍且研究最深入的问题之一。然而,用于评估模型性能的指标却较少受到关注。受试者工作特征曲线下面积(AUROC)长期以来一直是模型比较的标准选择。尽管其优点显著,但对局部类别交换不变性(LxC)问题并不理想,本文首次提出该类不变性概念。为此,我们提出LxCIM(LxC不变指标),该指标不仅基于排序且对局部类别交换不变,还具备直观性、逻辑一致性及始终可计算的优点,并可通过累积准确率-决策率曲线实现更深入的分析。此外,LxCIM与AUROC、准确率及准确率-决策率曲线下面积(AUDRC)存在明确的理论联系,可从多个角度理解:作为对称形式的AUROC、准确率的排序类比,或更代表性和可解释的AUDRC变体。最后,我们展示了LxCIM在双变量因果发现问题中的直接应用(该问题具有局部类别交换不变性),并证明其克服了现有指标在此领域的公认局限。所有代码和实现细节均公开于github.com/tiagobrogueira/Causal-Discovery-In-Exchangeable-Data。
原文摘要 · Abstract (English)
Binary classification is one of the oldest, most prevalent, and studied problems in machine learning. However, the metrics used to evaluate model performance have received comparatively little attention. The area under the receiver operating characteristic curve (AUROC) has long been a standard choice for model comparison. Despite its advantages, AUROC is not always ideal, particularly for problems that are invariant to local exchange of classes (LxC), a new form of metric invariance introduced in this work. To address this limitation, we propose LxCIM (LxC-invariant metric), which is not only rank-based and invariant under local exchange of classes, but also intuitive, logically consistent, and always computable, while enabling more detailed analysis through the cumulative accuracy-decision rate curve. Moreover, LxCIM exhibits clear theoretical connections to AUROC, accuracy, and the area under the accuracy-decision rate curve (AUDRC). These relationships allow for multiple complementary interpretations: as a symmetric form of AUROC, a rank-based analogue of accuracy, or a more representative and more interpretable variant of AUDRC. Finally, we demonstrate the direct applicability of LxCIM to the bivariate causal discovery problem (which exhibits invariance to local exchange of classes) and show how it addresses the acknowledged limitations of existing metrics used in this field. All code and implementation details are publicly available at github.com/tiagobrogueira/Causal-Discovery-In-Exchangeable-Data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。