用自监督学习提升超声视频分割,仅需少量标注数据即可显著提升效果。
Self-Supervised Ultrasound-Video Segmentation with Feature Prediction and 3D Localised Loss
- 通过特征预测+3D局部化任务,提升ViT模型对超声视频的局部理解能力。
- 在仅10%标注数据下,分割性能提升达8.35%,100%数据下提升3.4%。
- 适合缺乏标注数据的医疗图像分割场景,尤其适用于超声视频分析。
由于超声成像对比度低、噪声大且易受伪影影响,获取和标注大规模数据集极具挑战,耗时且需专业临床知识。自监督学习(SSL)可通过未标注数据学习有效表征,在标注数据有限时提升分割性能。近期最先进的视频自监督框架V-JEPA仅基于特征预测,避免像素级重建或负样本,更适合超声影像——对噪声敏感度低,同时充分利用时间信息。据我们所知,这是首个将V-JEPA应用于超声视频的研究。与VideoMAE等基于补丁掩码的SSL方法类似,V-JEPA适用于基于ViT的模型。然而,ViTs在小型医学数据集上表现不佳,因缺乏归纳偏置、空间局部性不足,且无层次特征学习机制。为此,我们提出一种新颖的3D局部化辅助任务,增强预训练阶段中ViT表示的局部性。结果表明,采用该辅助任务的V-JEPA在多种冻结编码器配置下均显著提升分割性能:使用100%训练数据时提升最高达3.4%,仅使用10%数据时提升高达8.35%。
原文摘要 · Abstract (English)
Acquiring and annotating large datasets in ultrasound imaging is challenging due to low contrast, high noise, and susceptibility to artefacts. This process requires significant time and clinical expertise. Self-supervised learning (SSL) offers a promising solution by leveraging unlabelled data to learn useful representations, enabling improved segmentation performance when annotated data is limited. Recent state-of-the-art developments in SSL for video data include V-JEPA, a framework solely based on feature prediction, avoiding pixel level reconstruction or negative samples. We hypothesise that V-JEPA is well-suited to ultrasound imaging, as it is less sensitive to noisy pixel-level detail while effectively leveraging temporal information. To the best of our knowledge, this is the first study to adopt V-JEPA for ultrasound video data. Similar to other patch-based masking SSL techniques such as VideoMAE, V-JEPA is well-suited to ViT-based models. However, ViTs can underperform on small medical datasets due to lack of inductive biases, limited spatial locality and absence of hierarchical feature learning. To improve locality understanding, we propose a novel 3D localisation auxiliary task to improve locality in ViT representations during V-JEPA pre-training. Our results show V-JEPA with our auxiliary task improves segmentation performance significantly across various frozen encoder configurations, with gains up to 3.4\% using 100\% and up to 8.35\% using only 10\% of the training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。