arXiv:2603.05867cs.CV2026-03中稿 · ICLR被引 2

用多模态链式思维分析肿瘤,提升诊断可追溯性与准确性。

TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis

  • 构建跨影像、报告到病理的链式推理框架,实现多轮自修正。
  • 在150万条标注数据上验证,病理分类准确率显著优于基线。
  • 适合临床辅助诊断与医学AI可解释性研究者使用。

精准肿瘤分析是临床放射学与精准肿瘤学的核心,早期检测、可靠病灶表征及病理级风险评估直接影响诊疗决策。链式思维(CoT)在此场景中尤为重要,能实现从影像发现到临床印象与病理结论的逐步推理解读,提升可追溯性并减少误诊。本文针对临床肿瘤分析任务,构建了一个大规模基准,涵盖从发现、印象到病理预测的多模态推理流程。我们整理了包含150万条带链式推理标注的VQA指令与3D CT扫描的TumorCoT数据集,提供逐步对齐的推理路径与跨模态一致性支持,可用于评估答案准确性与推理连贯性。进一步提出TumorChain框架,通过3D影像编码器、临床文本理解与器官级视觉-语言对齐的紧密耦合,实现跨模态对齐与迭代式因果推理。该模型在多轮自精炼中融合视觉证据,聚合结论并输出病理预测,显著提升可追溯性并降低幻觉风险。实验表明,其在病灶检测、印象生成与病理分类任务上持续优于强基线,在DeepTumorVQA基准上展现良好泛化能力。结果表明,多模态链式推理在临床实践中的可靠与可解释肿瘤分析具有巨大潜力。项目详情见:https://github.com/ZJU4HealthCare/TumorChain。

原文摘要 · Abstract (English)

Accurate tumor analysis is central to clinical radiology and precision oncology, where early detection, reliable lesion characterization, and pathology-level risk assessment guide diagnosis and treatment planning. Chain-of-Thought (CoT) reasoning is particularly important in this setting because it enables step-by-step interpretation from imaging findings to clinical impressions and pathology conclusions, improving traceability and reducing diagnostic errors. Here, we target the clinical tumor analysis task and build a large-scale benchmark that operationalizes a multimodal reasoning pipeline, spanning findings, impressions, and pathology predictions. We curate TumorCoT, a large-scale dataset of 1.5M CoT-labeled VQA instructions paired with 3D CT scans, with step-aligned rationales and cross-modal alignments along the trajectory from findings to impression to pathology, enabling evaluation of both answer accuracy and reasoning consistency. We further propose TumorChain, a multimodal interleaved reasoning framework that tightly couples 3D imaging encoders, clinical text understanding, and organ-level vision-language alignment. Through cross-modal alignment and iterative interleaved causal reasoning, TumorChain grounds visual evidence, aggregates conclusions, and issues pathology predictions after multiple rounds of self-refinement, improving traceability and reducing hallucination risk. Experiments show consistent improvements over strong baselines in lesion detection, impression generation, and pathology classification, and demonstrate strong generalization on the DeepTumorVQA benchmark. These results highlight the potential of multimodal reasoning for reliable and interpretable tumor analysis in clinical practice. Detailed information about our project can be found on our project homepage at https://github.com/ZJU4HealthCare/TumorChain.

肿瘤分析链式推理多模态可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。