用多模态切片发现技术自动检测医学图像分类的系统性失败
A multimodal slice discovery framework for systematic failure detection and explanation in medical image classification
- 基于多模态表示扩展切片发现方法
- 在MIMIC-CXR-JPG数据集上验证了故障发现与解释能力
- 适合医疗AI安全审计，资源受限时也有效
尽管基于机器学习的医学图像分类器取得进展，其安全性和可靠性在实际应用中仍是重大挑战。现有审计方法主要依赖单模态特征或基于元数据的子组分析，可解释性差，常无法捕捉隐藏的系统性故障。为此，我们提出首个针对医疗应用的自动化审计框架，将切片发现方法扩展至多模态表示。在常见故障场景下，基于MIMIC-CXR-JPG数据集的全面实验表明，该框架具备强大的故障发现与解释生成能力。结果还显示，多模态信息通常能实现更全面有效的审计，而仅使用图像以外的单模态变体在资源受限场景下也展现出显著潜力。
原文摘要 · Abstract (English)
Despite advances in machine learning-based medical image classifiers, the safety and reliability of these systems remain major concerns in practical settings. Existing auditing approaches mainly rely on unimodal features or metadata-based subgroup analyses, which are limited in interpretability and often fail to capture hidden systematic failures. To address these limitations, we introduce the first automated auditing framework that extends slice discovery methods to multimodal representations specifically for medical applications. Comprehensive experiments were conducted under common failure scenarios using the MIMIC-CXR-JPG dataset, demonstrating the framework's strong capability in both failure discovery and explanation generation. Our results also show that multimodal information generally allows more comprehensive and effective auditing of classifiers, while unimodal variants beyond image-only inputs exhibit strong potential in scenarios where resources are constrained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。