一个能看多种医学影像的通用模型，准确率最高达98.8%。
Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)
- 用自监督学习和记忆模块训练，跨10个专科7种模态数据
- 在7种影像上平均准确率超85%，最高达98.8%（OCT）
- 适合医疗多模态分析，尤其适合缺乏标注数据的场景
当前医学影像人工智能模型大多仅针对单一模态和疾病。尝试构建多模态、多病种模型时，临床准确性常不稳定。且训练通常需大量人工标注数据。我们开发了MerMED-FM，一种先进的多模态、多专科基础模型，采用自监督学习和记忆模块进行训练。该模型在超过十种专科、七种模态的330万张医学图像上训练，包括计算机断层扫描（CT）、胸部X光（CXR）、超声（US）、病理切片、彩色眼底摄影（CFP）、光学相干断层扫描（OCT）及皮肤图像。在多个疾病上评估并对比现有基础模型，表现优异：所有模态均表现良好，奥氏曲线下面积（AUROC）分别为：OCT 0.988；病理 0.982；超声 0.951；CT 0.943；皮肤 0.931；CFP 0.894；CXR 0.858。MerMED-FM有望成为高度可适配、多功能的跨专科基础模型，支持多样医学领域的可靠影像解读。
原文摘要 · Abstract (English)
Current artificial intelligence models for medical imaging are predominantly single modality and single disease. Attempts to create multimodal and multi-disease models have resulted in inconsistent clinical accuracy. Furthermore, training these models typically requires large, labour-intensive, well-labelled datasets. We developed MerMED-FM, a state-of-the-art multimodal, multi-specialty foundation model trained using self-supervised learning and a memory module. MerMED-FM was trained on 3.3 million medical images from over ten specialties and seven modalities, including computed tomography (CT), chest X-rays (CXR), ultrasound (US), pathology patches, color fundus photography (CFP), optical coherence tomography (OCT) and dermatology images. MerMED-FM was evaluated across multiple diseases and compared against existing foundational models. Strong performance was achieved across all modalities, with AUROCs of 0.988 (OCT); 0.982 (pathology); 0.951 (US); 0.943 (CT); 0.931 (skin); 0.894 (CFP); 0.858 (CXR). MerMED-FM has the potential to be a highly adaptable, versatile, cross-specialty foundation model that enables robust medical imaging interpretation across diverse medical disciplines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。