用AI自动生成阿尔茨海默病影像报告，提升诊断效率。
Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease
- 用GPT-4o-mini从结构化数据生成合成诊断报告
- 基于BiomedCLIP和T5模型直接从影像生成报告，指标达BLEU-4 0.1827
- 为缺乏报告的神经影像研究提供新方法，适合医学AI开发者
大型语言模型（LLMs）和视觉语言模型（VLMs）在医学影像诊断中展现出巨大潜力，尤其在放射科，其数据常与人工诊断报告配对。然而，阿尔茨海默病等神经影像领域因缺乏可用于模型微调的完整诊断报告而存在显著研究空白。本文通过GPT-4o-mini对663名患者的OASIS-4数据集结构化信息生成合成诊断报告，并以这些合成报告为真实标签，训练并验证了基于预训练BiomedCLIP与T5模型的影像报告生成系统。实验结果表明，该方法在生成报告上达到BLEU-4 0.1827、ROUGE-L 0.3719、METEOR 0.4163的得分，展现出生成临床相关且准确报告的潜力。
原文摘要 · Abstract (English)
The rapid advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have shown great potential in medical diagnostics, particularly in radiology, where datasets such as X-rays are paired with human-generated diagnostic reports. However, a significant research gap exists in the neuroimaging field, especially for conditions such as Alzheimer's disease, due to the lack of comprehensive diagnostic reports that can be utilized for model fine-tuning. This paper addresses this gap by generating synthetic diagnostic reports using GPT-4o-mini on structured data from the OASIS-4 dataset, which comprises 663 patients. Using the synthetic reports as ground truth for training and validation, we then generated neurological reports directly from the images in the dataset leveraging the pre-trained BiomedCLIP and T5 models. Our proposed method achieved a BLEU-4 score of 0.1827, ROUGE-L score of 0.3719, and METEOR score of 0.4163, revealing its potential in generating clinically relevant and accurate diagnostic reports.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。