AI和专家在皮肤镜图像上都犯错，说明图片本身有内在模糊性。
When AI and Experts Agree on Error: Intrinsic Ambiguity in Dermatoscopic Images
- 用多种CNN模型找出被普遍误判的图像子集。
- 专家对这些难图的诊断一致性降至kappa=0.08，远低于对照组的0.61。
- 图像质量是导致人机共同出错的关键因素，适合医学AI可信性研究者参考。
将人工智能（尤其是卷积神经网络，CNN）融入皮肤科诊断展现出显著临床潜力。现有文献多以人类专家为基准评估算法性能，而本研究采用新视角，探究皮肤镜图像本身的内在复杂性。通过多组CNN架构的严格实验，我们识别出一组被所有模型系统性误判的图像，其错误率显著高于随机水平。为判断失败源于算法偏差还是视觉固有模糊性，多位皮肤科专家独立评估这些难题与对照组图像。结果显示，在AI误判的图像上，专家诊断表现严重下降：与金标准的一致性大幅降低，Cohen's kappa从对照组的0.61降至0.08；医生间共识也急剧恶化，Fleiss kappa由对照组的0.456降至0.275。我们发现图像质量是导致人机双重失效的主要驱动因素。为提升透明度与可复现性，所有数据、代码及训练模型均已公开。
原文摘要 · Abstract (English)
The integration of artificial intelligence (AI), particularly Convolutional Neural Networks (CNNs), into dermatological diagnosis demonstrates substantial clinical potential. While existing literature predominantly benchmarks algorithmic performance against human experts, our study adopts a novel perspective by investigating the intrinsic complexity of dermatoscopic images. Through rigorous experimentation with multiple CNN architectures, we isolated a subset of images systematically misclassified across all models-a phenomenon statistically proven to exceed random chance. To determine if these failures stem from algorithmic biases or inherent visual ambiguity, expert dermatologists independently evaluated these challenging cases alongside a control group. The results revealed a collapse in human diagnostic performance on the AI-misclassified images. First, agreement with ground-truth labels plummeted, with Cohen's kappa dropping to a mere 0.08 for the difficult images, compared to a 0.61 for the control group. Second, we observed a severe deterioration in expert consensus; inter-rater reliability among physicians fell from moderate concordance (Fleiss kappa = 0.456) on control images to only modest agreement (Fleiss kappa = 0.275) on difficult cases. We identified image quality as a primary driver of these dual systematic failures. To promote transparency and reproducibility, all data, code, and trained models have been made publicly available
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。