arXiv:2605.15720cs.CVcs.LG2026-05

用跨模态对齐增强半监督医学图像指代分割,提升弱标注场景下的精度。

Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment

论文配图:Semi-MedRef: Semi-Supervised Medical Referring Image Segmentation with Cross-Modal Alignment
图 1 · 摘自论文原文
  • 设计T-PatchMix与位置感知文本增强,保持图像与语言的对应关系。
  • 在QaTa-COV19和MosMedData+上,不同标注比例下均超越现有方法。
  • 适合医疗视觉-语言联合学习、标注成本高的研究者使用。

医学指代图像分割(MRIS)需从医学图像中预测病灶掩码并匹配自然语言描述,但像素级标注与文本配对成本高。半监督学习(SSL)可通过利用未标注数据缓解此问题,但其效果依赖于强扰动下图像-文本对齐的保持。现有方法多采用独立或简单跨模态扰动(如左右翻转),而像CutMix这类更强的数据增强因可能破坏跨模态对应关系而未被充分探索。本文提出Semi-MedRef,一种教师-学生框架,通过三个互补组件实现对齐:T-PatchMix,一种同步图像块混合与位置语言、伪掩码更新的对齐保持增强;PosAug,一种位置感知文本增强,防止模型过度依赖位置表达;以及位置亲和对比学习(PACL),利用粗粒度位置线索构建解剖加权的软正样本,促进解剖结构驱动的跨模态表示学习。在QaTa-COV19与MosMedData+数据集上的实验表明,Semi-MedRef在所有标注比例下均持续优于最先进的全监督与半监督MRIS方法。

原文摘要 · Abstract (English)

Medical referring image segmentation (MRIS) predicts lesion masks from medical images and natural-language referring expressions, but acquiring paired pixel-level annotations and referring texts is costly. Semi-supervised learning (SSL) can alleviate this burden by exploiting unlabeled data, yet its effectiveness depends on preserving image--text alignment under strong perturbations. Existing SSL methods for referring segmentation rely on independent or simple multimodal perturbations (e.g., left--right flips), while stronger augmentations such as CutMix remain largely unexplored because they can disrupt cross-modal correspondence. We propose Semi-MedRef, a teacher--student SSL framework that explicitly preserves alignment between medical images and positional language through three complementary components: T-PatchMix, an alignment-preserving cross-modal augmentation that synchronizes patch mixing with positional-language and pseudo-mask updates; PosAug, a position-aware text augmentation that regularizes reliance on positional expressions; and Positional Affinity Contrastive Learning (PACL), which exploits coarse positional cues to construct region-aware supervision through anatomically weighted soft positives, encouraging anatomically grounded cross-modal representation learning. Experiments on QaTa-COV19 and MosMedData+ demonstrate that Semi-MedRef consistently outperforms state-of-the-art fully supervised and semi-supervised MRIS methods across all label regimes.

医学图像指代分割半监督跨模态对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。