通过器官区域信息增强，提升医学影像报告生成的准确性与鲁棒性。
ORID: Organ-Regional Information Driven Framework for Radiology Report Generation
- 基于器官区域构建跨模态融合模块，精准整合影像与诊断描述信息。
- 引入图神经网络分析器官重要性系数，降低无关器官噪声干扰。
- 在多个评估指标上超越当前最优方法，适合临床辅助诊断场景。
放射科报告生成（RRG）旨在基于医学影像自动输出连贯的疾病分析文本，减轻放射科医生的工作负担。现有AI方法主要集中在编码器-解码器架构的改进。本文提出一种器官区域信息驱动（ORID）框架，有效整合多模态信息并减少无关器官带来的噪声影响。首先，在LLaVA-Med基础上构建针对RRG的指令数据集，提升器官区域诊断描述能力，得到LLaVA-Med-RRG。随后，设计基于器官的跨模态融合模块，有效结合器官区域诊断描述与影像信息。为进一步抑制无关器官噪声对报告生成的影响，提出器官重要性系数分析模块，利用图神经网络（GNN）分析各器官区域跨模态信息间的关联。大量实验及与先进方法的对比表明，所提方法在多种评估指标上表现更优。
原文摘要 · Abstract (English)
The objective of Radiology Report Generation (RRG) is to automatically generate coherent textual analyses of diseases based on radiological images, thereby alleviating the workload of radiologists. Current AI-based methods for RRG primarily focus on modifications to the encoder-decoder model architecture. To advance these approaches, this paper introduces an Organ-Regional Information Driven (ORID) framework which can effectively integrate multi-modal information and reduce the influence of noise from unrelated organs. Specifically, based on the LLaVA-Med, we first construct an RRG-related instruction dataset to improve organ-regional diagnosis description ability and get the LLaVA-Med-RRG. After that, we propose an organ-based cross-modal fusion module to effectively combine the information from the organ-regional diagnosis description and radiology image. To further reduce the influence of noise from unrelated organs on the radiology report generation, we introduce an organ importance coefficient analysis module, which leverages Graph Neural Network (GNN) to examine the interconnections of the cross-modal information of each organ region. Extensive experiments an1d comparisons with state-of-the-art methods across various evaluation metrics demonstrate the superior performance of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。