构建模拟放射科医生诊断流程的肺部X光评估基准,提升AI可解释性。
Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning
- 设计多阶段视觉问答数据集,还原真实读片推理链条
- 模型在1100万问答对上训练,跨5个外部数据集表现更优
- 适合医疗AI可解释性研究者与临床辅助系统开发者
基于人工智能的胸部X光(CXR)解读助手已取得显著进展并逐步应用于临床。然而,当前医学AI模型多采用简单的输入-输出范式,直接处理图像与指令生成结果,指令常嵌入模型架构,忽视了放射科诊断中固有的推理过程。这种推理通常是序列化的,每个阶段需结合图像、任务目标及前序阶段的上下文信息。该忽略导致模型与临床场景脱节、上下文缺失且错误难以追溯。为此,我们构建了首个模拟真实临床诊断推理流程的多阶段视觉问答(VQA)数据集CXRTrek,涵盖8个连续诊断阶段,共428,966个样本和超过1100万组问题-答案对,平均每个样本包含26.29组问答。基于此数据集,我们提出专门适配临床推理流程的视觉语言大模型CXRTrekNet,能有效建模各诊断阶段间的依赖关系,并捕捉放射学语境下的推理模式。在自建数据集上训练后,该模型在多个任务上持续优于现有医学视觉语言大模型,并在五个外部数据集上展现出更强泛化能力。相关数据集与模型已在开源仓库(https://github.com/guanjinquan/CXRTrek)发布。
原文摘要 · Abstract (English)
Artificial intelligence (AI)-based chest X-ray (CXR) interpretation assistants have demonstrated significant progress and are increasingly being applied in clinical settings. However, contemporary medical AI models often adhere to a simplistic input-to-output paradigm, directly processing an image and an instruction to generate a result, where the instructions may be integral to the model's architecture. This approach overlooks the modeling of the inherent diagnostic reasoning in chest X-ray interpretation. Such reasoning is typically sequential, where each interpretive stage considers the images, the current task, and the contextual information from previous stages. This oversight leads to several shortcomings, including misalignment with clinical scenarios, contextless reasoning, and untraceable errors. To fill this gap, we construct CXRTrek, a new multi-stage visual question answering (VQA) dataset for CXR interpretation. The dataset is designed to explicitly simulate the diagnostic reasoning process employed by radiologists in real-world clinical settings for the first time. CXRTrek covers 8 sequential diagnostic stages, comprising 428,966 samples and over 11 million question-answer (Q&A) pairs, with an average of 26.29 Q&A pairs per sample. Building on the CXRTrek dataset, we propose a new vision-language large model (VLLM), CXRTrekNet, specifically designed to incorporate the clinical reasoning flow into the VLLM framework. CXRTrekNet effectively models the dependencies between diagnostic stages and captures reasoning patterns within the radiological context. Trained on our dataset, the model consistently outperforms existing medical VLLMs on the CXRTrek benchmarks and demonstrates superior generalization across multiple tasks on five diverse external datasets. The dataset and model can be found in our repository (https://github.com/guanjinquan/CXRTrek).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。