用普通照片+患者信息提升皮肤癌检测准确率,无需专业设备。
Multimodal system for skin cancer detection
- 融合照片与患者信息的多模态模型,分两步处理有无元数据情况。
- 在不平衡数据下仍达0.18068的局部ROC AUC和78.4%的前15检索敏感度。
- 适合资源有限或缺乏皮肤镜的基层医疗机构使用。
黑色素瘤的早期检测对诊断与治疗至关重要。尽管基于皮肤镜图像的深度学习模型已展现潜力,但其依赖专用设备,限制了在更广泛临床环境中的应用。本研究提出一种基于常规照片的多模态黑色素瘤检测系统,提升了可及性与灵活性。系统整合图像数据与患者人口统计学、病灶特征等表格型元数据,采用多模态神经网络联合处理图像与元数据,并支持有无元数据的两阶段模型。进一步通过三阶段流程优化预测,引入增强算法以提升性能。针对高度不平衡的数据集,采用了特定训练策略确保模型鲁棒性。消融实验评估了多种视觉架构、提升算法与损失函数,最终取得最高0.18068的局部ROC AUC(理论最大0.2)和0.78371的前15项检索敏感度。结果表明,在结构化多阶段流程中融合照片与元数据可显著提升检测性能。该系统提供了一种可扩展、无需专业设备的解决方案,适用于多样化的医疗环境,弥合了专科与通用临床实践之间的差距。
原文摘要 · Abstract (English)
Melanoma detection is vital for early diagnosis and effective treatment. While deep learning models on dermoscopic images have shown promise, they require specialized equipment, limiting their use in broader clinical settings. This study introduces a multi-modal melanoma detection system using conventional photo images, making it more accessible and versatile. Our system integrates image data with tabular metadata, such as patient demographics and lesion characteristics, to improve detection accuracy. It employs a multi-modal neural network combining image and metadata processing and supports a two-step model for cases with or without metadata. A three-stage pipeline further refines predictions by boosting algorithms and enhancing performance. To address the challenges of a highly imbalanced dataset, specific techniques were implemented to ensure robust training. An ablation study evaluated recent vision architectures, boosting algorithms, and loss functions, achieving a peak Partial ROC AUC of 0.18068 (0.2 maximum) and top-15 retrieval sensitivity of 0.78371. Results demonstrate that integrating photo images with metadata in a structured, multi-stage pipeline yields significant performance improvements. This system advances melanoma detection by providing a scalable, equipment-independent solution suitable for diverse healthcare environments, bridging the gap between specialized and general clinical practices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。