用视频版CLIP模型，少样本预测心脏射血分数。
CardiacCLIP: Video-based CLIP Adaptation for LVEF Prediction in a Few-shot Manner
- 用注意力机制选关键帧，多尺度提取心肌结构特征
- 1样本下在EchoNet-Dynamic数据集上误差降低2.07%
- 适合标注数据少的临床场景，提升模型泛化能力
超声心动图是评估心脏功能的重要无创手段，左室射血分数（LVEF）是衡量心功能的关键指标。现有LVEF估计方法依赖大规模标注视频数据集，成本高且难以适应不同临床环境。现有基于视觉-语言的超声模型（如EchoCLIP）采用图像到文本的预训练，但未能捕捉重要时序动态和局部心脏结构信息。为此，我们提出CardiacCLIP，一种基于视频的框架，通过注意力帧融合与多分辨率输入缩放提升LVEF预测性能。具体地，引入MFL（多帧学习）机制，通过注意力选择关键帧进行信息融合；提出EchoZoom多尺度特征提取策略，增强心脏结构的空间表征。作为CLIP模型在少样本超声视频分析中的新适配，本方法显著提升诊断准确率，在EchoNet-Dynamic数据集1样本设置下将平均绝对误差（MAE）降低2.07%。代码已开源：https://github.com/xmed-lab/CardiacCLIP。
原文摘要 · Abstract (English)
Echocardiography is a vital non-invasive modality for cardiac assessment, with left ventricular ejection fraction (LVEF) serving as a key indicator of heart function. Existing LVEF estimation methods depend on large-scale annotated video datasets, which are costly and limit adaptability across various clinical settings. Recent vision-language models for echocardiography, such as EchoCLIP, apply image-to-text pretraining but fail to capture crucial temporal dynamics and localized cardiac structures essential for accurate diagnosis. To address these challenges, we propose CardiacCLIP, a video-based framework that enhances LVEF prediction through attention-based frame aggregation and multi-resolution input scaling. Specifically, we introduce MFL (Multi Frame Learning), a novel attention-based mechanism for selectively fusing informative frames, and EchoZoom, a multi-scale feature extraction strategy that refines spatial representations of cardiac structures. As a novel adaptation of CLIP models for few-shot echocardiogram video analysis, our approach significantly improves diagnostic accuracy, reducing MAE by 2.07 on the EchoNet-Dynamic dataset under 1-shot setting. The code is available at https://github.com/xmed-lab/CardiacCLIP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。