用放射科报告训练肿瘤分割模型,显著提升准确率。
Report Supervision

- 用报告描述的肿瘤数量、大小和位置指导分割模型训练
- 在仅50个标注掩码时,检测F1分数提升15%
- 适合数据少但报告多的医疗影像场景
分割模型在肿瘤检测中可超越放射科医生、分类模型和视觉语言模型。重要的是,分割模型能勾勒出肿瘤边界,帮助放射科医生验证并信任AI输出。其主要限制在于肿瘤掩码稀缺:创建一个3D肿瘤掩码耗时长达30分钟,因此大多数公开CT数据集仅包含数百个掩码,即使最大的私有数据集也仅有数千个。肿瘤掩码并非临床常规生成,但放射科报告却大量存在。公开数据集包含数万对CT-报告,医院则有数十万对。这些报告详细描述肿瘤特征,提供大规模、高信息量的训练数据。本文提出报告监督(R-Super)训练框架,直接利用报告监督并改进肿瘤分割模型。R-Super引入新损失函数,使分割模型学习生成与报告描述一致的肿瘤数量、大小和位置。报告仅用于训练。我们在肾癌和胰腺癌分割任务上评估R-Super,涵盖最多41,418对CT-报告及3,488个胰腺肿瘤掩码的数据量。外部验证显示,相比仅用掩码训练,R-Super将肿瘤检测F1分数和分割骰子系数(DSC)最高提升15%。其表现优于CLIP和多任务学习等方法。利用大量易得的报告补充稀缺掩码,R-Super在极少数训练掩码(如50个)和充足掩码(如3,488个)情况下均显著提升性能,推动肿瘤分割迈向规模化。
原文摘要 · Abstract (English)
Segmentation models can surpass radiologists, classification models, and vision-language models in tumor detection. Importantly, segmentation models outline tumors, allowing radiologists to better verify and trust the AI output. Their main limitation is the scarcity of tumor masks: creating one 3D tumor mask takes up to 30 minutes, so most public CT datasets contain only a few hundred masks, and even the largest private datasets contain only a couple of thousand. Tumor masks are not produced in clinical routine, but radiology reports are. Public datasets contain tens of thousands of CT-Report pairs, and hospitals contain hundreds of thousands. These reports describe tumors in detail, providing large-scale, informative training data. Here, we introduce Report Supervision (R-Super), a training framework that uses reports to directly supervise and improve tumor segmentation. R-Super introduces new loss functions that teach segmentation models to segment tumors that match report descriptions of tumor count, sizes, and locations. Reports are only used for training. We evaluated R-Super on kidney and pancreatic tumor segmentation, exploring diverse training data sizes, up to 41,418 CT-Report plus 3,488 pancreatic tumor CT-Mask pairs. On external validation, R-Super increased tumor detection F1-Score and segmentation DSC by up to +15% with respect to mask-only training. It also surpassed alternative methods such as CLIP and multi-task learning. Leveraging numerous readily available reports to supplement scarce masks, R-Super strongly improves AI performance when very few training masks are available (e.g., 50), and when many masks are available (e.g., 3,488), unlocking scale in tumor segmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。