用可解释的医学概念增强多模态检索生成,让放射报告更准更可信。
Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generation
- 将影像分解为可解释的临床概念,融合到多模态检索生成中
- 在MIMIC-CXR和IU X-Ray上提升诊断准确率与文本质量
- 适合需要可信赖AI辅助诊断的医疗AI研发与临床落地
通过视觉语言模型(VLM)生成放射科报告有望减轻文档负担、提高报告一致性并加速临床流程。然而,其临床应用受限于缺乏可解释性以及与影像证据不符的幻觉问题。现有研究通常将可解释性与准确性视为独立目标:基于概念的解释方法侧重透明度,而检索增强生成(RAG)方法则通过外部检索提升事实准确性。本文提出概念增强型多模态RAG(CEMRAG),将视觉表示分解为可解释的临床概念,并将其与多模态RAG融合,生成更具上下文信息的提示,从而同时提升可解释性与事实准确性。在MIMIC-CXR和IU X-Ray数据集上,针对多种VLM架构、训练策略与检索配置的实验表明,该方法在临床准确性指标与标准NLP评估中均显著优于传统RAG及仅使用概念的基线模型。结果挑战了可解释性与性能之间的权衡假设,表明透明的视觉概念可增强而非损害诊断准确性。模块化设计将可解释性拆分为视觉透明性与结构化语言模型条件,为构建临床可信的AI辅助放射学提供可遵循路径。
原文摘要 · Abstract (English)
Radiology Report Generation (RRG) through Vision-Language Models (VLMs) promises to reduce documentation burden, improve reporting consistency, and accelerate clinical workflows. However, their clinical adoption remains limited by the lack of interpretability and the tendency to hallucinate findings misaligned with imaging evidence. Existing research typically treats interpretability and accuracy as separate objectives, with concept-based explainability techniques focusing primarily on transparency, while Retrieval-Augmented Generation (RAG) methods targeting factual grounding through external retrieval. We present Concept-Enhanced Multimodal RAG (CEMRAG), a unified framework that decomposes visual representations into interpretable clinical concepts and integrates them with multimodal RAG. This approach exploits enriched contextual prompts for RRG, improving both interpretability and factual accuracy. Experiments on MIMIC-CXR and IU X-Ray across multiple VLM architectures, training regimes, and retrieval configurations demonstrate consistent improvements over both conventional RAG and concept-only baselines on clinical accuracy metrics and standard NLP measures. These results challenge the assumed trade-off between interpretability and performance, showing that transparent visual concepts can enhance rather than compromise diagnostic accuracy in medical VLMs. Our modular design decomposes interpretability into visual transparency and structured language model conditioning, providing a principled pathway toward clinically trustworthy AI-assisted radiology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。