评估两款皮肤癌诊断模型的可解释性与公平性,发现对深色皮肤人群效果较差。
Evaluating Machine Learning-based Skin Cancer Diagnosis
- 用显著图和积分梯度分析模型决策依据,由皮肤科医生验证。
- 深色皮肤群体的误诊率差异显著,尤其是漏诊情况更严重。
- 通过校准后处理策略降低偏差,改善深肤色人群的诊断公平性。
本研究评估了两种基于深度学习的皮肤癌检测模型的可靠性,重点关注其可解释性与公平性。利用包含10,000张皮肤镜图像的HAM10000数据集,评估了MobileNet基线模型与自定义CNN模型在七类皮肤病变分类及良恶性区分上的表现。通过显著图与积分梯度进行可解释性分析,并由皮肤科医生解读结果。结果显示,两类模型对多数病变类型均能突出相关特征,但对脂溢性角化病与血管性病变识别较弱。公平性使用等几率(Equalized Odds)指标在性别与肤色分组中评估:两模型在性别间表现公平,但在浅色与深色皮肤间存在显著的假阳性与假阴性率差异。采用校准后的等几率后处理策略可有效缓解偏差,尤其降低了假阴性率差异。研究结论指出,尽管模型具备一定可解释性,仍需改进以确保跨肤色群体的公平性。该研究强调医疗AI需在多样化人群中进行严格评估。
原文摘要 · Abstract (English)
This study evaluates the reliability of two deep learning models for skin cancer detection, focusing on their explainability and fairness. Using the HAM10000 dataset of dermatoscopic images, the research assesses two convolutional neural network architectures: a MobileNet-based model and a custom CNN model. Both models are evaluated for their ability to classify skin lesions into seven categories and to distinguish between dangerous and benign lesions. Explainability is assessed using Saliency Maps and Integrated Gradients, with results interpreted by a dermatologist. The study finds that both models generally highlight relevant features for most lesion types, although they struggle with certain classes like seborrheic keratoses and vascular lesions. Fairness is evaluated using the Equalized Odds metric across sex and skin tone groups. While both models demonstrate fairness across sex groups, they show significant disparities in false positive and false negative rates between light and dark skin tones. A Calibrated Equalized Odds postprocessing strategy is applied to mitigate these disparities, resulting in improved fairness, particularly in reducing false negative rate differences. The study concludes that while the models show promise in explainability, further development is needed to ensure fairness across different skin tones. These findings underscore the importance of rigorous evaluation of AI models in medical applications, particularly in diverse population groups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。