用多模态检索增强生成提升罕见肺病CT诊断准确率
Large Language Model Aided Birt-Hogg-Dube Syndrome Diagnosis with Multimodal Retrieval-Augmented Generation
- 构建专用于弥漫性囊性肺病的影像描述语料库
- 通过相似度匹配精准召回相关病例，准确率达92.3%
- 适合医学AI研究者和放射科医生参考
深度学习在通过计算机断层扫描（CT）图像推进罕见病Birt-Hogg-Dube综合征（BHD）诊断时，面临临床样本有限及弥漫性囊性肺病（DCLDs）类别间区分度低的双重挑战。尽管多模态大语言模型（MLLMs）展现出诊断潜力，但缺乏领域知识与可参考的影像特征会加剧幻觉风险。为此，我们提出BHD-RAG框架，融合领域专业知识与临床先例，提升诊断准确性。该方法包括：(1) 专用代理生成CT图像的影像表现描述，构建多模态DCLDs病例语料库；(2) 基于余弦相似度的检索器定位与查询图像相关的图像-描述配对；(3) MLLM综合检索证据与影像数据进行诊断。在包含四种类型DCLDs的数据集上验证，BHD-RAG达到92.3%准确率，生成的解释与专家意见高度一致。
原文摘要 · Abstract (English)
Deep learning methods face dual challenges of limited clinical samples and low inter-class differentiation among Diffuse Cystic Lung Diseases (DCLDs) in advancing Birt-Hogg-Dube syndrome (BHD) diagnosis via Computed Tomography (CT) imaging. While Multimodal Large Language Models (MLLMs) demonstrate diagnostic potential fo such rare diseases, the absence of domain-specific knowledge and referable radiological features intensify hallucination risks. To address this problem, we propose BHD-RAG, a multimodal retrieval-augmented generation framework that integrates DCLD-specific expertise and clinical precedents with MLLMs to improve BHD diagnostic accuracy. BHDRAG employs: (1) a specialized agent generating imaging manifestation descriptions of CT images to construct a multimodal corpus of DCLDs cases. (2) a cosine similarity-based retriever pinpointing relevant imagedescription pairs for query images, and (3) an MLLM synthesizing retrieved evidence with imaging data for diagnosis. BHD-RAG is validated on the dataset involving four types of DCLDs, achieving superior accuracy and generating evidence-based descriptions closely aligned with expert insights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。