arXiv:2605.15997cs.CV2026-05

统一框架实现CT图像分割、检测与解释,支持临床级细粒度分析。

Segmentation, Detection and Explanation: A Unified Framework for CT Appearance Reasoning

论文配图:Segmentation, Detection and Explanation: A Unified Framework for CT Appearance Reasoning
图 1 · 摘自论文原文
  • 用任务路由令牌驱动视觉语言模型生成分割掩码和边界框。
  • 在BTCV和MosMed+上分别提升1.0%和1.7% Dice,优于现有方法。
  • 适合医学影像分析、AI辅助诊断系统研发者使用。

深度学习虽推动了CT图像分析发展,但多局限于图像级模式识别,缺乏明确解剖或上下文推理能力。尽管大视觉语言模型引入语言上下文,但多数方法仅针对单一任务,难以满足临床工作流中对多种细粒度分析的需求。本文提出一种统一的自回归框架,将语言引导的视觉推理融入CT解读。通过任务路由令牌触发检测与分割头,基于大视觉语言模型的隐藏状态生成视觉输出(如掩码、边界框)与文本推理。为提升定位精度与语义清晰度,设计了“更近观察”机制,实现从粗到细的区域逐步聚焦。构建了一个新的多模态CT数据集,包含像素级掩码、边界框、空间提示与结构化描述,采用AI辅助标注结合人工验证。在公开基准测试中表现优异,于BTCV和MosMed+上分别获得1.0%与1.7%的Dice提升,同时输出外观推理结果。代码与数据集将开源。

原文摘要 · Abstract (English)

Recent progress in deep learning has significantly advanced CT image analysis, particularly for segmentation tasks. However, these advances are largely confined to image-level pattern recognition, with most methods lacking explicit anatomical or contextual reasoning. Large vision-language models introduce linguistic context into image analysis, yet most approaches typically focus on a single task, which is insufficient for clinical workflow analysis that requires multiple fine-grained types of analysis, such as anatomy detection and segmentation. In this paper, we propose a unified autoregressive framework that integrates language-guided visual reasoning into CT interpretation. Our method introduces task-routing tokens that trigger detection and segmentation heads conditioned on the hidden states of a large vision-language model, enabling coherent generation of visual outputs (e.g., masks and bounding boxes) and textual reasonings. To progressively enhance localisation accuracy and semantic clarity, we further design a "closer-look" mechanism that allows the model to perform progressive coarse-to-fine visits to regions of interest under refined fields of view. To support model training and evaluation, we curated a new multimodal CT dataset containing pixel-wise masks, bounding boxes, spatial prompts, and structured descriptions for visual objects constructed through an AI-assisted annotation process with human verification. Experiments on public benchmarks demonstrate consistent improvements over the SoTA, achieving up to 1.0% Dice on BTCV and 1.7% Dice on MosMed+, while additionally providing appearance reasoning outputs. The code and dataset will be available.

CT分析视觉推理多任务医学影像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。