通过在线聚类蒸馏,让心脏超声视频学习更精准的时空表征。
Self-supervised Learning of Echocardiographic Video Representations via Online Cluster Distillation
- 双分支结构:视频分支捕捉时序动态,图像分支提取精细空间语义。
- 在6个数据集上零样本和线性探测均超越现有方法,实现优异分割迁移与LVEF预测性能。
- 适合医学影像、超声分析及自监督学习研究者,尤其关注临床任务应用。
自监督学习(SSL)在自然图像与视频理解中取得显著进展,但在心脏超声等医学领域仍面临挑战,原因包括解剖结构细微、时间动态复杂,且缺乏领域专用预训练模型。现有对比、掩码建模及聚类方法在高样本相似性、低信噪比输入或剧烈增强导致临床特征失真等问题上表现不佳。本文提出DISCOVR(基于跨模态视频表征的蒸馏图像监督),一种自监督双分支框架,用于心脏超声视频表征学习。该框架结合基于聚类的视频编码器以建模时序动态,以及在线图像编码器以提取细粒度空间语义。两者通过语义聚类蒸馏损失连接,将演化中的图像编码器知识传递至视频编码器,从而生成具备时间连贯性与细粒度语义理解的表征。在涵盖胎儿、儿科与成人人群的六个超声心动图数据集上评估,DISCOVR在零样本与线性探测设置下优于专门的视频异常检测方法及先进视频自监督基线,实现更优的分割迁移效果,并在左室射血分数(LVEF)预测等临床任务中表现出色。代码已开源:https://github.com/mdivyanshu97/DISCOVR
原文摘要 · Abstract (English)
Self-supervised learning (SSL) has achieved major advances in natural images and video understanding, but challenges remain in domains like echocardiography (heart ultrasound) due to subtle anatomical structures, complex temporal dynamics, and the current lack of domain-specific pre-trained models. Existing SSL approaches such as contrastive, masked modeling, and clustering-based methods struggle with high intersample similarity, sensitivity to low PSNR inputs common in ultrasound, or aggressive augmentations that distort clinically relevant features. We present DISCOVR (Distilled Image Supervision for Cross Modal Video Representation), a self-supervised dual branch framework for cardiac ultrasound video representation learning. DISCOVR combines a clustering-based video encoder that models temporal dynamics with an online image encoder that extracts fine-grained spatial semantics. These branches are connected through a semantic cluster distillation loss that transfers anatomical knowledge from the evolving image encoder to the video encoder, enabling temporally coherent representations enriched with fine-grained semantic understanding.Evaluated on six echocardiography datasets spanning fetal, pediatric, and adult populations, DISCOVR outperforms both specialized video anomaly detection methods and state-of-the-art video-SSL baselines in zero-shot and linear probing setups,achieving superior segmentation transfer and strong downstream performance on clinically relevant tasks such as LVEF prediction. Code available at: https://github.com/mdivyanshu97/DISCOVR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。