找出模型表现好或差的特定数据子群体，提升可信度。
Conformalized Exceptional Model Mining: Telling Where Your Model Performs (Not) Well
- 用置信预测+异常模型挖掘，定位性能异常的数据子集。
- 提出新指标RAUL，识别多分类与回归中的异常性能模式。
- 适合关注模型可靠性与可解释性的研究人员和工程师。
理解机器学习模型的细微性能表现对负责任部署至关重要，尤其在医疗、金融等高风险领域。本文提出一种新框架——共形化异常模型挖掘（Conformalized Exceptional Model Mining），结合置信预测的严谨性与异常模型挖掘（EMM）的解释力，识别数据中模型表现显著偏离的连贯子群体，揭示高置信与高不确定性区域。我们开发了新型模型类mSMoPE（多路软模型性能评估），通过置信预测的严格覆盖保证量化不确定性。定义新质量指标相对平均不确定性损失（RAUL），在多分类与回归任务中分离出具有异常性能特征的子群体。跨多个数据集的实验表明，该框架能有效发现可解释的子群体，为模型行为提供关键洞察。本工作推动了可解释人工智能与不确定性量化的技术前沿。
原文摘要 · Abstract (English)
Understanding the nuanced performance of machine learning models is essential for responsible deployment, especially in high-stakes domains like healthcare and finance. This paper introduces a novel framework, Conformalized Exceptional Model Mining, which combines the rigor of Conformal Prediction with the explanatory power of Exceptional Model Mining (EMM). The proposed framework identifies cohesive subgroups within data where model performance deviates exceptionally, highlighting regions of both high confidence and high uncertainty. We develop a new model class, mSMoPE (multiplex Soft Model Performance Evaluation), which quantifies uncertainty through conformal prediction's rigorous coverage guarantees. By defining a new quality measure, Relative Average Uncertainty Loss (RAUL), our framework isolates subgroups with exceptional performance patterns in multi-class classification and regression tasks. Experimental results across diverse datasets demonstrate the framework's effectiveness in uncovering interpretable subgroups that provide critical insights into model behavior. This work lays the groundwork for enhancing model interpretability and reliability, advancing the state-of-the-art in explainable AI and uncertainty quantification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。