arXiv:2510.09953cs.CV2025-10

通过对比检索引导的联合优化,让分割模型像人一样从参考图中学习。

J-RAS: Mutual Adaptation for Medical Image Segmentation via Contrastive Retrieval-Augmented Joint Optimization

  • 用对比学习让检索模型主动找相关图像-掩码对,反哺分割模型
  • 在ACDC数据集上,SegFormer的Dice得分提升至0.9115,边界误差下降
  • 适合缺乏标注数据或跨数据集泛化困难的医学图像分割任务

临床医生手动分割医学图像虽准确但耗时且存在专家差异;而基于AI的模型在数据有限和领域漂移下表现不佳。受人类学习新任务时依赖参考图启发,我们提出联合检索增强分割框架J-RAS,通过交替进行对比学习与监督学习,联合优化分割模型与检索模型。检索模型可发现上下文相关的图像-掩码对,提升分割模型的解剖结构推理能力。不同于传统被动提供相似样本的检索增强方法,J-RAS建立双向自适应优化机制:检索模型学会强调分割相关线索,分割模型则利用检索样例改善边界分割、稀有病例鲁棒性及跨数据集泛化能力。在两个公开基准ACDC与M&Ms上,使用U-Net、TransUNet、SAM和SegFormer等多种主干网络验证了J-RAS的通用性与有效性。例如,在ACDC上,SegFormer的平均Dice由0.8708±0.042提升至0.9115±0.031,豪斯多夫距离由1.8130±2.49降至1.1489±0.30。结果表明,检索引导的对比优化能有效融合人类指导与机器精度。

原文摘要 · Abstract (English)

Manual medical image segmentation by clinicians, though accurate, is time-consuming and variable across experts, while AI-based models automate this process but often falter under limited data and domain shifts. Inspired by how humans learn new tasks through guidance such as how children draw more accurately when shown reference images, we propose Joint Retrieval-Augmented Segmentation (J-RAS), a framework that enables segmentation networks to learn with guidance. J-RAS jointly optimizes a segmentation model and a retrieval model through alternating contrastive and supervised learning, allowing the retrieval network to discover contextually relevant image-mask pairs that refine the segmentation model's anatomical reasoning. Unlike conventional retrieval-based augmentation that passively provides similar samples, J-RAS establishes a mutual adaptation and optimization loop where the retrieval model learns to emphasize segmentation-relevant cues, while the segmentation model leverages retrieved examples to improve boundary delineation, robustness to rare cases and and cross-dataset generalization. Evaluations on two public benchmarks, ACDC and M&Ms, across multiple backbones (U-Net, TransUNet, SAM, and SegFormer) demonstrate the generality and effectiveness of J-RAS. For instance, on ACDC, SegFormer improves from a mean Dice of 0.8708$\pm$0.042 and HD of 1.8130$\pm$2.49 to 0.9115$\pm$0.031 and 1.1489$\pm$0.30. These results highlight how retrieval-guided contrastive optimization bridges human-like guidance and machine-learned precision in medical image segmentation.

医学图像分割对比学习检索增强自适应优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。