arXiv:2411.11943cs.CVcs.AI2024-11被引 13

首个可控制生成疾病进展视频的框架,实现个性化模拟。

Medical Video Generation for Disease Progression Simulation

  • 用大模型重写提示词,规划疾病轨迹
  • 多轮扩散模型生成真实中间状态序列
  • 适合临床医生、医学教育及数据补全

疾病进展建模对提升临床诊断与预后至关重要,但常因缺乏个体患者的纵向医学影像监测而受限。为此,我们提出首个医学视频生成(MVG)框架,支持对疾病相关图像与视频特征的可控操作,实现精确、真实且个性化的疾病进展模拟。方法首先利用大语言模型(LLMs)重写疾病轨迹提示词;随后,采用可控多轮扩散模型为每位患者模拟疾病进展状态,生成逼真的中间状态序列;最后,通过基于扩散的视频过渡生成模型,在这些状态间进行插值。我们在胸部X光、眼底摄影和皮肤图像三个医学影像领域验证了该框架。结果表明,MVG在生成连贯且临床上合理的疾病轨迹方面显著优于基线模型。两名资深医师的用户研究进一步验证了生成序列的临床价值。MVG有望辅助医疗人员建模疾病轨迹、填补缺失影像数据,并通过动态可视化提升医学教育效果。

原文摘要 · Abstract (English)

Modeling disease progression is crucial for improving the quality and efficacy of clinical diagnosis and prognosis, but it is often hindered by a lack of longitudinal medical image monitoring for individual patients. To address this challenge, we propose the first Medical Video Generation (MVG) framework that enables controlled manipulation of disease-related image and video features, allowing precise, realistic, and personalized simulations of disease progression. Our approach begins by leveraging large language models (LLMs) to recaption prompt for disease trajectory. Next, a controllable multi-round diffusion model simulates the disease progression state for each patient, creating realistic intermediate disease state sequence. Finally, a diffusion-based video transition generation model interpolates disease progression between these states. We validate our framework across three medical imaging domains: chest X-ray, fundus photography, and skin image. Our results demonstrate that MVG significantly outperforms baseline models in generating coherent and clinically plausible disease trajectories. Two user studies by veteran physicians, provide further validation and insights into the clinical utility of the generated sequences. MVG has the potential to assist healthcare providers in modeling disease trajectories, interpolating missing medical image data, and enhancing medical education through realistic, dynamic visualizations of disease progression.

医学影像视频生成扩散模型疾病模拟

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。