arXiv:2507.03585cs.CVcs.AI2025-07中稿 · IEEE ICASSP 2026被引 6

用大模型做因果推理,让医学图像分割更鲁棒、可交互。

Causal-SAM-LLM: Large Language Models as Causal Reasoners for Robust Medical Segmentation

  • 让大模型分析影像风格,剥离无关特征,提升泛化能力。
  • 测试时通过自然语言指令实时修正分割结果,误差降低15.8毫米。
  • 只需不到9%参数量,就能超越现有最强模型,适合临床部署。

深度学习模型在医学图像分割中的临床应用受限于其对未见领域的泛化能力不足,根源常在于模型学习了解剖结构与特定成像风格之间的虚假关联。为此,我们提出Causal-SAM-LLM框架,将大语言模型(LLMs)作为因果推理器。该框架基于冻结的Segment Anything Model(SAM)编码器,引入两项协同创新:首先,语言对抗解耦(LAD)利用视觉-语言模型生成丰富的风格文本描述,训练分割模型特征与这些描述对比不相似,从而去除非因果信息;其次,测试时因果干预(TCI)提供交互机制,由大模型解析临床医生的自然语言指令,实时调节分割解码器特征,实现精准错误修正。我们在四个公开数据集(BTCV、CHAOS、AMOS、BraTS)组成的综合基准上进行评估,涵盖跨扫描仪、跨模态和跨解剖结构设置。Causal-SAM-LLM在分布外(OOD)泛化性能上达到新标杆,平均Dice分数提升最高达6.2点,豪斯多夫距离减少15.8毫米,且仅使用全模型不到9%的可训练参数。本工作为构建鲁棒、高效、可交互的医疗AI系统开辟了新路径。

原文摘要 · Abstract (English)

The clinical utility of deep learning models for medical image segmentation is severely constrained by their inability to generalize to unseen domains. This failure is often rooted in the models learning spurious correlations between anatomical content and domain-specific imaging styles. To overcome this fundamental challenge, we introduce Causal-SAM-LLM, a novel framework that elevates Large Language Models (LLMs) to the role of causal reasoners. Our framework, built upon a frozen Segment Anything Model (SAM) encoder, incorporates two synergistic innovations. First, Linguistic Adversarial Disentanglement (LAD) employs a Vision-Language Model to generate rich, textual descriptions of confounding image styles. By training the segmentation model's features to be contrastively dissimilar to these style descriptions, it learns a representation robustly purged of non-causal information. Second, Test-Time Causal Intervention (TCI) provides an interactive mechanism where an LLM interprets a clinician's natural language command to modulate the segmentation decoder's features in real-time, enabling targeted error correction. We conduct an extensive empirical evaluation on a composite benchmark from four public datasets (BTCV, CHAOS, AMOS, BraTS), assessing generalization under cross-scanner, cross-modality, and cross-anatomy settings. Causal-SAM-LLM establishes a new state of the art in out-of-distribution (OOD) robustness, improving the average Dice score by up to 6.2 points and reducing the Hausdorff Distance by 15.8 mm over the strongest baseline, all while using less than 9% of the full model's trainable parameters. Our work charts a new course for building robust, efficient, and interactively controllable medical AI systems.

医学分割因果推理大模型可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。