用语言视觉模型辅助肺癌肿瘤勾画,显著降低误判率。
A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncology
- 结合CT图像与临床文本描述,用大模型减少假阳性
- 假阳性减少72.4%,F1得分0.652,误检率降35.0%
- 适合放射肿瘤学、AI医疗研发人员参考
肺癌是全球癌症死亡的首要原因。肿瘤勾画的复杂性对放疗至关重要,但专业人才在资源有限地区常不可得。人工智能(AI)借助深度学习和自然语言处理技术提供解决方案,但仍面临高假阳性问题。本文提出放疗勾画助手(Oncology Contouring Copilot, OCC),利用资深医生的文本描述,通过语言视觉模型(LVMs)如GPT-4V,融合文本与影像数据,自动完成肿瘤轮廓勾画。系统先从CT扫描中识别结节候选区域,再通过临床描述文本有效降低假阳性。部署结果显示,假发现率下降35.0%,每幅扫描假阳性减少72.4%,在数据集上实现F1-score 0.652。OCC通过整合专家知识,提升勾画质量,优化放疗流程,提出新型医学语言视觉提示技术以抑制大模型幻觉,并开展多模型对比分析,验证其在医疗语言视觉任务中的潜力。
原文摘要 · Abstract (English)
Background: Lung cancer ranks as the leading cause of cancer-related mortality worldwide. The complexity of tumor delineation, crucial for radiation therapy, requires expertise often unavailable in resource-limited settings. Artificial Intelligence(AI), particularly with advancements in deep learning (DL) and natural language processing (NLP), offers potential solutions yet is challenged by high false positive rates. Purpose: The Oncology Contouring Copilot (OCC) system is developed to leverage oncologist expertise for precise tumor contouring using textual descriptions, aiming to increase the efficiency of oncological workflows by combining the strengths of AI with human oversight. Methods: Our OCC system initially identifies nodule candidates from CT scans. Employing Language Vision Models (LVMs) like GPT-4V, OCC then effectively reduces false positives with clinical descriptive texts, merging textual and visual data to automate tumor delineation, designed to elevate the quality of oncology care by incorporating knowledge from experienced domain experts. Results: Deployments of the OCC system resulted in a significant reduction in the false discovery rate by 35.0%, a 72.4% decrease in false positives per scan, and an F1-score of 0.652 across our dataset for unbiased evaluation. Conclusions: OCC represents a significant advance in oncology care, particularly through the use of the latest LVMs to improve contouring results by (1) streamlining oncology treatment workflows by optimizing tumor delineation, reducing manual processes; (2) offering a scalable and intuitive framework to reduce false positives in radiotherapy planning using LVMs; (3) introducing novel medical language vision prompt techniques to minimize LVMs hallucinations with ablation study, and (4) conducting a comparative analysis of LVMs, highlighting their potential in addressing medical language vision challenges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。