用图像生成视频模拟呼吸运动,提升4D医学影像时间一致性。
Temporal Differential Fields for 4D Motion Modeling via Image-to-Video Synthesis

- 通过图像到视频合成,从首帧预测后续帧的动态变化。
- 在ACDC和4D Lung数据集上,生成视频与真实运动轨迹高度一致。
- 设计时序差分场增强模型对细微运动的捕捉能力,适合医学影像动态建模。
规律性呼吸运动的时间建模对图像引导临床应用至关重要。现有方法需同时获取起始与终止帧的高剂量扫描才能模拟时间动态,但术前采集中患者微小移动会导致呼吸周期首尾帧间出现难以通过图像配准消除的动态背景偏差,影响时间建模效果。为此,我们首次提出基于图像到视频(I2V)合成框架,仅用首帧即可预测指定长度的未来帧。为提升生成视频的时间一致性,设计了时序差分扩散模型,生成相邻帧间的相对差分表示。引入提示注意力层以精细捕捉差分特征,并采用场增强层使差分信息更好地融入I2V框架,从而更准确地刻画合成视频的时间变化。在ACDC心脏与4D Lung数据集上的大量实验表明,该方法能沿内在运动轨迹生成4D视频,在感知相似性和时间一致性方面优于其他先进方法。代码将很快公开。
原文摘要 · Abstract (English)
Temporal modeling on regular respiration-induced motions is crucial to image-guided clinical applications. Existing methods cannot simulate temporal motions unless high-dose imaging scans including starting and ending frames exist simultaneously. However, in the preoperative data acquisition stage, the slight movement of patients may result in dynamic backgrounds between the first and last frames in a respiratory period. This additional deviation can hardly be removed by image registration, thus affecting the temporal modeling. To address that limitation, we pioneeringly simulate the regular motion process via the image-to-video (I2V) synthesis framework, which animates with the first frame to forecast future frames of a given length. Besides, to promote the temporal consistency of animated videos, we devise the Temporal Differential Diffusion Model to generate temporal differential fields, which measure the relative differential representations between adjacent frames. The prompt attention layer is devised for fine-grained differential fields, and the field augmented layer is adopted to better interact these fields with the I2V framework, promoting more accurate temporal variation of synthesized videos. Extensive results on ACDC cardiac and 4D Lung datasets reveal that our approach simulates 4D videos along the intrinsic motion trajectory, rivaling other competitive methods on perceptual similarity and temporal consistency. Codes will be available soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。