arXiv:2501.13247cs.LGcs.CV2025-01被引 1

用图像和病历判断慢性伤口是否需转诊,帮家庭护士做更准决策。

Multimodal AI on Wound Images and Clinical Notes for Home Patient Referral

  • 融合视觉与文本特征,用ViT和DeBERTa分析伤口照片和病历
  • 在小而不平衡数据上达77%准确率和70%F1值,优于已有方法
  • 通过可解释性工具让推荐理由透明,适合临床辅助使用

慢性伤口影响约850万美国人,尤其老年人与糖尿病患者。这些伤口可能长达九个月才能愈合,定期护理至关重要,以避免截肢等严重后果。许多患者由访视护士在家中接受护理,但护士专业水平参差,导致护理质量不一。应转诊至专科医生的非愈合伤口常被错误、延迟或不必要的处理。本文提出深度多模态伤口评估工具(DM-WAT),一种机器学习框架,帮助访视护士判断是否转诊慢性伤口患者。DM-WAT分析智能手机拍摄的伤口图像及电子健康记录(EHR)中的临床笔记。它采用DeiT-Base-Distilled(视觉变压器)提取图像特征,用DeBERTa-base提取文本特征,并通过中间融合方式结合二者。针对小样本且不平衡的数据挑战,引入图像与文本增强及迁移学习,实现高性能表现。评估显示,DM-WAT达到77%(标准差3%)准确率与70%(标准差2%)F1分数,优于先前方法。使用Score-CAM与Captum解释算法揭示影响推荐的关键图像与文本区域,提升可解释性与信任度。

原文摘要 · Abstract (English)

Chronic wounds affect 8.5 million Americans, particularly the elderly and patients with diabetes. These wounds can take up to nine months to heal, making regular care essential to ensure healing and prevent severe outcomes like limb amputations. Many patients receive care at home from visiting nurses with varying levels of wound expertise, leading to inconsistent care. Problematic, non-healing wounds should be referred to wound specialists, but referral decisions in non-clinical settings are often erroneous, delayed, or unnecessary. This paper introduces the Deep Multimodal Wound Assessment Tool (DM-WAT), a machine learning framework designed to assist visiting nurses in deciding whether to refer chronic wound patients. DM-WAT analyzes smartphone-captured wound images and clinical notes from Electronic Health Records (EHRs). It uses DeiT-Base-Distilled, a Vision Transformer (ViT), to extract visual features from images and DeBERTa-base to extract text features from clinical notes. DM-WAT combines visual and text features using an intermediate fusion approach. To address challenges posed by a small and imbalanced dataset, it integrates image and text augmentation with transfer learning to achieve high performance. In evaluations, DM-WAT achieved 77% with std 3% accuracy and a 70% with std 2% F1 score, outperforming prior approaches. Score-CAM and Captum interpretation algorithms provide insights into specific parts of image and text inputs that influence recommendations, enhancing interpretability and trust.

多模态医疗AI伤口评估可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。