通过不确定性建模提升医学影像与文本联合分割的准确性与可靠性
Uncertainty-Aware Vision-Language Segmentation for Medical Imaging
- 设计轻量级跨模态融合模块，实现图像与文本高效对齐
- 在低质量图像下仍保持高分割精度，相较SOTA提速30%以上
- 适合临床场景中数据模糊或不完整时的智能诊断应用
我们提出一种新型不确定性感知的多模态分割框架，结合放射科图像与相关临床文本，实现精准医疗诊断。引入模态解码注意力块（MoDAB）与轻量级状态空间混合器（SSMix），实现高效的跨模态融合与长距离依赖建模。为应对模糊性，提出谱熵不确定性（SEU）损失函数，统一建模空间重叠、光谱一致性和预测不确定性。在图像质量较差的复杂临床场景中显著提升模型可靠性。在多个公开医学数据集QATA-COVID19、MosMed++和Kvasir-SEG上的实验表明，该方法在分割性能上优于现有最先进方法，且计算效率显著更高。结果凸显了不确定性建模与结构化模态对齐在视觉-语言医学分割任务中的重要性。
原文摘要 · Abstract (English)
We introduce a novel uncertainty-aware multimodal segmentation framework that leverages both radiological images and associated clinical text for precise medical diagnosis. We propose a Modality Decoding Attention Block (MoDAB) with a lightweight State Space Mixer (SSMix) to enable efficient cross-modal fusion and long-range dependency modelling. To guide learning under ambiguity, we propose the Spectral-Entropic Uncertainty (SEU) Loss, which jointly captures spatial overlap, spectral consistency, and predictive uncertainty in a unified objective. In complex clinical circumstances with poor image quality, this formulation improves model reliability. Extensive experiments on various publicly available medical datasets, QATA-COVID19, MosMed++, and Kvasir-SEG, demonstrate that our method achieves superior segmentation performance while being significantly more computationally efficient than existing State-of-the-Art (SoTA) approaches. Our results highlight the importance of incorporating uncertainty modelling and structured modality alignment in vision-language medical segmentation tasks. Code: https://github.com/arya-domain/UA-VLS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。