用大模型+视觉语言模型自动发现腹部CT的意外病灶,提升报告效率与准确率。
INFORM-CT: INtegrating LLMs and VLMs FOR Incidental Findings Management in Abdominal CT
- 通过计划-执行框架,用大模型生成检测脚本,视觉模型执行分析
- 在三个腹部器官上实现端到端自动检测,准确率优于纯视觉模型方法
- 适合临床辅助诊断系统研发者及医学影像工程师参考
腹部CT中的意外发现虽多为良性,但可能具有重要临床意义,应按指南报告。传统人工阅片耗时且主观性强。本文提出一种新框架,利用大语言模型(LLMs)和基础视觉语言模型(VLMs),采用计划-执行智能体方式,自动完成腹部CT中意外发现的检测、分类与报告。基于腹部器官的医学指南,该框架由规划器(基于LLM生成预定义函数的Python脚本)与执行器(调用VLM、分割模型与图像处理子程序运行脚本)构成。在三个器官的腹部CT基准测试中,实现了完全自动的端到端验证。结果表明,该框架在准确率与效率方面均优于现有纯VLM方法。
原文摘要 · Abstract (English)
Incidental findings in CT scans, though often benign, can have significant clinical implications and should be reported following established guidelines. Traditional manual inspection by radiologists is time-consuming and variable. This paper proposes a novel framework that leverages large language models (LLMs) and foundational vision-language models (VLMs) in a plan-and-execute agentic approach to improve the efficiency and precision of incidental findings detection, classification, and reporting for abdominal CT scans. Given medical guidelines for abdominal organs, the process of managing incidental findings is automated through a planner-executor framework. The planner, based on LLM, generates Python scripts using predefined base functions, while the executor runs these scripts to perform the necessary checks and detections, via VLMs, segmentation models, and image processing subroutines. We demonstrate the effectiveness of our approach through experiments on a CT abdominal benchmark for three organs, in a fully automatic end-to-end manner. Our results show that the proposed framework outperforms existing pure VLM-based approaches in terms of accuracy and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。