通过分析影像模态冲突,让医学图像分割更可信。
EnTrust: Modeling Inter-Modal Conflict for Trustworthy Multimodal Medical Image Analysis

- 将多模态影像差异分解为共识、特异和冲突三部分,结构化建模不确定性。
- 在四个临床领域中分割精度达顶尖水平,校准误差降低40%。
- 只需单模型即可媲美五倍深度集成,适合临床部署的高效可靠系统。
多模态医学影像融合解剖与功能信息,但病灶异质区域常出现模态间不一致。现有分割模型要么简单平均消除分歧,要么事后估算不确定性,均无法解释预测不可靠的原因。本文提出EnTrust框架,将模态间冲突视为预测不确定性的核心来源。其EnFuse模块将多模态特征解耦为三部分:共享解剖共识(F_c)、模态特异性线索(F_{u,m})和空间局部冲突信号(F_{cf}),通过交叉协方差目标强制独立性。该结构化分解引导SegDiff——一种基于扩散的生成分割模型,使其在模态分歧区域生成差异化的假设。TrustMap则利用集合熵、冲突引导扰动探测和可学习校准头,将假设分歧转化为校准后的像素级不确定性,使临床医生不仅知道哪里不确定,还理解原因。在涵盖脑部、心脏、病灶与肿瘤领域的四个基准上,EnTrust实现最先进分割精度,校准误差比最强基线降低40%。值得注意的是,其性能超越5倍深度集成,仅需单模型且内存占用约为一半。代码与模型检查点见https://github.com/GenMI-Lab/EnTrust.git。
原文摘要 · Abstract (English)
Multimodal medical imaging fuses complementary anatomical and functional information, yet modalities frequently disagree in pathologically heterogeneous regions. Current segmentation models handle this in one of two inadequate ways: deterministic fusion that averages away disagreement, or post-hoc uncertainty estimation decoupled from the fusion process that produces it. Both obscure the clinically critical question: why is this prediction unreliable? We present EnTrust, a framework that treats inter-modal conflict as the primary source of predictive uncertainty. Our EnFuse module decomposes multimodal features into three disentangled components: shared anatomical consensus (F_c), modality-specific cues (F_{u,m}), and spatially localized conflict signals (F_{cf}), with independence enforced via a cross-covariance objective. This structured decomposition conditions SegDiff, a diffusion-based generative segmentation model whose sampled hypotheses diverge specifically in regions of modal disagreement. TrustMap then translates this hypothesis divergence into calibrated, pixel-wise uncertainty using ensemble entropy, conflict-guided perturbation probing, and a learned calibration head, enabling clinicians to understand not only where predictions are uncertain, but why. Across four benchmarks spanning brain, cardiac, lesion, and oncology domains, EnTrust achieves state-of-the-art segmentation accuracy while reducing calibration error by 40% compared to the strongest baseline. Notably, it outperforms 5x deep ensembles using a single model at roughly half the memory footprint. Code and checkpoints are available at https://github.com/GenMI-Lab/EnTrust.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。