用心电图生成心脏超声视频,提升自动化诊断效率。
ECHOPulse: ECG controlled echocardio-grams video generation
- 以心电图为条件生成超声视频,无需专家标注。
- 采用分词与掩码建模,生成速度大幅提升。
- 可推广至心脏磁共振等多模态影像生成。
心脏超声(ECHO)对心脏评估至关重要,但其视频质量与解读高度依赖人工经验,导致临床与便携设备结果不一致。通过合成数据生成高质量超声视频可实现自动化监测。现有模型普遍存在计算成本高、推理慢及依赖复杂条件提示的问题。为此,我们提出ECHOPULSE,一种基于心电图(ECG)条件的超声视频生成模型。ECHOPULSE引入两项关键改进:(1)利用VQ-VAE分词与掩码视觉分词实现快速解码;(2)直接以易获取的时序心电信号作为条件,避免复杂提示需求。据我们所知,这是首个使用时序信号如心电图进行超声视频生成的工作。ECHOPULSE不仅支持可控的合成数据生成,还可提供超越心电图的动态心脏功能信息,用于疾病监测与预测。在三个公开及私有数据集上的评估显示,其在定性与定量指标上均达到当前最优表现。此外,该模型可轻松扩展至心脏磁共振、功能磁共振及3D CT等其他模态生成任务。
原文摘要 · Abstract (English)
Echocardiography (ECHO) is essential for cardiac assessments, but its video quality and interpretation heavily relies on manual expertise, leading to inconsistent results from clinical and portable devices. ECHO video generation offers a solution by improving automated monitoring through synthetic data and generating high-quality videos from routine health data. However, existing models often face high computational costs, slow inference, and rely on complex conditional prompts that require experts' annotations. To address these challenges, we propose ECHOPULSE, an ECG-conditioned ECHO video generation model. ECHOPULSE introduces two key advancements: (1) it accelerates ECHO video generation by leveraging VQ-VAE tokenization and masked visual token modeling for fast decoding, and (2) it conditions on readily accessible ECG signals, which are highly coherent with ECHO videos, bypassing complex conditional prompts. To the best of our knowledge, this is the first work to use time-series prompts like ECG signals for ECHO video generation. ECHOPULSE not only enables controllable synthetic ECHO data generation but also provides updated cardiac function information for disease monitoring and prediction beyond ECG alone. Evaluations on three public and private datasets demonstrate state-of-the-art performance in ECHO video generation across both qualitative and quantitative measures. Additionally, ECHOPULSE can be easily generalized to other modality generation tasks, such as cardiac MRI, fMRI, and 3D CT generation. Demo can seen from \url{https://github.com/levyisthebest/ECHOPulse_Prelease}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。