arXiv:2409.09520cs.CVcs.AI2024-09中稿 · WACV 2025被引 8

用SAM自动发现皮肤病变视觉概念,提升诊断可解释性。

Enhancing Skin Disease Diagnosis: Interpretable Visual Concept Discovery with SAM

  • 结合SAM与跨注意力融合,从临床照片中提取可解释的视觉概念
  • 在两个数据集上实现高精度诊断,且能生成清晰的病变定位图
  • 适合需要可解释性医疗AI的医生和研究者使用

当前AI辅助皮肤图像诊断已达到皮肤科医生水平,但在实际应用中仍面临标注数据少、图像变化大及需详细解释以保障患者安全等挑战。传统分割方法依赖像素级标注,而现有SAM应用多局限于边界清晰的皮肤镜图像,难以处理手机拍摄的非标准化临床照片。为此,我们提出一种跨注意力融合框架,利用SAM通过简单提示生成皮肤病变的视觉概念,并将局部特征与全局图像信息融合,提升诊断性能与可解释性。在两个皮肤疾病数据集上的实验表明,该方法在病变诊断准确率和可视化解释能力上均表现优异。

原文摘要 · Abstract (English)

Current AI-assisted skin image diagnosis has achieved dermatologist-level performance in classifying skin cancer, driven by rapid advancements in deep learning architectures. However, unlike traditional vision tasks, skin images in general present unique challenges due to the limited availability of well-annotated datasets, complex variations in conditions, and the necessity for detailed interpretations to ensure patient safety. Previous segmentation methods have sought to reduce image noise and enhance diagnostic performance, but these techniques require fine-grained, pixel-level ground truth masks for training. In contrast, with the rise of foundation models, the Segment Anything Model (SAM) has been introduced to facilitate promptable segmentation, enabling the automation of the segmentation process with simple yet effective prompts. Efforts applying SAM predominantly focus on dermatoscopy images, which present more easily identifiable lesion boundaries than clinical photos taken with smartphones. This limitation constrains the practicality of these approaches to real-world applications. To overcome the challenges posed by noisy clinical photos acquired via non-standardized protocols and to improve diagnostic accessibility, we propose a novel Cross-Attentive Fusion framework for interpretable skin lesion diagnosis. Our method leverages SAM to generate visual concepts for skin diseases using prompts, integrating local visual concepts with global image features to enhance model performance. Extensive evaluation on two skin disease datasets demonstrates our proposed method's effectiveness on lesion diagnosis and interpretability.

皮肤诊断视觉概念可解释性SAM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。