arXiv:2503.14979cs.CV2025-03

仅用首帧标注实现医疗视频目标分割,减轻标注负担。

One-Shot Medical Video Object Segmentation via Temporal Contrastive Memory Networks

  • 构建时序对比记忆网络,捕捉相邻帧间关系
  • 在多模态数据集上达到领先性能,泛化能力强
  • 适合医疗视频分析中标签稀缺场景

视频目标分割对高效分析复杂医学视频至关重要,但面临数据稀缺和标注困难。我们提出单次医疗视频目标分割任务,即仅凭第一帧的掩码标注,完成整个视频的前景与背景像素分离。为此,我们设计时序对比记忆网络,包含图像与掩码编码器以学习特征表示,一个时序对比记忆库用于对齐相邻帧嵌入并推开远距离帧嵌入,显式建模帧间关系并存储特征;解码器融合编码图像特征与记忆读取结果进行分割。同时,我们收集了一个跨模态、多源的医疗视频数据集用于基准测试。大量实验表明,该方法在仅使用单个示例的情况下,对已见和未见结构均实现先进分割性能,展现出从少量标注中良好泛化的能力,凸显其缓解医学视频标注压力的潜力。代码已公开于 https://github.com/MedAITech/TCMN。

原文摘要 · Abstract (English)

Video object segmentation is crucial for the efficient analysis of complex medical video data, yet it faces significant challenges in data availability and annotation. We introduce the task of one-shot medical video object segmentation, which requires separating foreground and background pixels throughout a video given only the mask annotation of the first frame. To address this problem, we propose a temporal contrastive memory network comprising image and mask encoders to learn feature representations, a temporal contrastive memory bank that aligns embeddings from adjacent frames while pushing apart distant ones to explicitly model inter-frame relationships and stores these features, and a decoder that fuses encoded image features and memory readouts for segmentation. We also collect a diverse, multi-source medical video dataset spanning various modalities and anatomies to benchmark this task. Extensive experiments demonstrate state-of-the-art performance in segmenting both seen and unseen structures from a single exemplar, showing ability to generalize from scarce labels. This highlights the potential to alleviate annotation burdens for medical video analysis. Code is available at https://github.com/MedAITech/TCMN.

视频分割医学影像少样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。