发现分类模型在特定人群中的异常表现，助力安全部署与数据优化
SubROC: AUC-Based Discovery of Exceptional Subgroup Performance for Binary Classifiers
- 基于异常模型挖掘，用AUC搜索潜在的高/低性能子群体
- 支持多指标评估，可识别性别+婚姻状态等组合下的极端表现
- 提供统计显著性检验，适合医疗、金融等领域模型可信度分析
机器学习在医疗、经济等实际应用中日益普及，可能影响大量人群。然而，模型性能常呈现异质性，在某些子群体中表现异常（如女性且已婚者）——或过差或过优。识别这些子群体有助于判断模型是否可在某类人群中安全部署，或需补充训练数据。现有方法缺乏高效一致的搜索框架。为此，我们提出SubROC：一个开源、易用的框架，基于异常模型挖掘，可靠高效地发现分类模型在可解释子群体中的优势与短板。SubROC融合了常见评估指标（ROC和PR AUC）、高效的搜索空间剪枝技术以实现快速全遍历、对类别不平衡的控制、冗余模式调整及显著性检验。我们在多个数据集的对比分析及案例研究中展示了SubROC的实际价值。
原文摘要 · Abstract (English)
Machine learning (ML) is increasingly employed in real-world applications like medicine or economics, thus, potentially affecting large populations. However, ML models often do not perform homogeneously, leading to underperformance or, conversely, unusually high performance in certain subgroups (e.g., sex=female AND marital_status=married). Identifying such subgroups can support practical decisions on which subpopulation a model is safe to deploy or where more training data is required. However, an efficient and coherent framework for effective search is missing. Consequently, we introduce SubROC, an open-source, easy-to-use framework based on Exceptional Model Mining for reliably and efficiently finding strengths and weaknesses of classification models in the form of interpretable population subgroups. SubROC incorporates common evaluation measures (ROC and PR AUC), efficient search space pruning for fast exhaustive subgroup search, control for class imbalance, adjustment for redundant patterns, and significance testing. We illustrate the practical benefits of SubROC in case studies as well as in comparative analyses across multiple datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。