arXiv:2504.07415cs.CVcs.CL2025-04ACL被引 2

用关键词检索提升放射科报告生成准确率,减少幻觉。

RA-RRG: Multimodal Retrieval-Augmented Radiology Report Generation with Key Phrase Extraction

  • 从报告中提取关键临床短语,输入图像时检索匹配短语
  • 在检索短语约束下生成报告,使CheXbert指标达顶尖水平
  • 无需大规模训练,适合多视角报告生成,临床实用性强

自动化放射科报告生成(RRG)有望减轻放射科医生的工作负担,近年来多模态大语言模型(MLLMs)已实现对胸部X光片(CXR)的报告生成。然而现有MLLMs计算成本高、需大规模训练数据,且易产生幻觉内容,限制了实际部署。为此,我们提出RA-RRG——一种结合多模态检索与大语言模型(LLMs)的增强型报告生成框架,通过检索相关关键短语来降低幻觉并减少计算需求。该方法利用LLM从历史报告中提取临床关键短语,并根据输入图像检索对应短语;再以检索到的短语为条件引导生成,有效抑制幻觉同时保持优异生成性能。在MIMIC-CXR和IU X-ray数据集上的实验表明,其在CheXbert指标上达到当前最优,在RadGraph F1分数上与主流MLLMs相当。此外,该框架可自然扩展至多视角报告生成,通过聚合多图检索结果实现,凸显其在真实临床场景中的广泛适用性。代码已公开于https://github.com/deepnoid-ai/RA-RRG。

原文摘要 · Abstract (English)

Automated radiology report generation (RRG) holds potential to reduce the workload of radiologists, and recent advances in multimodal large language models (MLLMs) have enabled multimodal chest X-ray (CXR) report generation. However, existing MLLMs are computationally expensive, require large-scale training data, and may produce hallucinated content, limiting their practical deployment. To address these limitations, we propose RA-RRG, a retrieval-augmented RRG framework that combines multimodal retrieval with large language models (LLMs) to generate radiology reports while reducing hallucinations and computational demands. RA-RRG uses LLMs to extract clinically essential key phrases from radiology reports and retrieves relevant phrases given an input image. By conditioning LLMs on the retrieved phrases, RA-RRG effectively suppresses hallucinations while maintaining strong report generation performance. Experiments on the MIMIC-CXR and IU X-ray datasets show state-of-the-art results on CheXbert metrics and competitive RadGraph F1 scores compared to MLLMs. Furthermore, RA-RRG naturally generalizes to multi-view RRG by aggregating phrases retrieved from multiple images, highlighting its broad applicability to real-world clinical scenarios. Code is available at https://github.com/deepnoid-ai/RA-RRG.

报告生成多模态检索增强医学AI

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。