用深度学习融合多源X射线视频,提升超高速成像的分辨率与帧率。
Deep learning-based spatio-temporal fusion for high-fidelity ultra-high-speed x-ray radiography
- 设计时空融合网络,结合低分辨率高帧率与高分辨率低帧率视频
- 在两组数据集上平均信噪比达37.57dB和35.15dB,优于基线方法
- 适合需要高保真超高速成像的物理实验与材料研究者
全场超高速(UHS)X射线成像已广泛用于表征各类过程与现象。然而,通过联合采集不同配置的X射线视频来挖掘其潜力尚未被充分开发。本文提出一种基于深度学习的时空融合(STF)框架,融合两组互补的X射线图像序列,重建出高空间分辨率、高帧率且高保真的目标图像序列。采用迁移学习策略训练模型,并在两个独立的X射线数据集上,将所提框架的峰值信噪比(PSNR)、平均绝对差(AAD)和结构相似性(SSIM)与基线深度学习模型、贝叶斯融合框架及双三次插值法进行对比。该框架在多种输入帧间隔和图像噪声水平下均表现更优。当使用4倍空间分辨率更低的低分辨率(LR)序列中连续3帧,以及20倍帧率更低的高分辨率(HR)序列中另2帧时,平均PSNR分别达到37.57 dB和35.15 dB。结合合适的高速相机组合,该方法可显著提升超高速X射线成像实验的性能与科学价值。
原文摘要 · Abstract (English)
Full-field ultra-high-speed (UHS) x-ray imaging experiments have been well established to characterize various processes and phenomena. However, the potential of UHS experiments through the joint acquisition of x-ray videos with distinct configurations has not been fully exploited. In this paper, we investigate the use of a deep learning-based spatio-temporal fusion (STF) framework to fuse two complementary sequences of x-ray images and reconstruct the target image sequence with high spatial resolution, high frame rate, and high fidelity. We applied a transfer learning strategy to train the model and compared the peak signal-to-noise ratio (PSNR), average absolute difference (AAD), and structural similarity (SSIM) of the proposed framework on two independent x-ray datasets with those obtained from a baseline deep learning model, a Bayesian fusion framework, and the bicubic interpolation method. The proposed framework outperformed the other methods with various configurations of the input frame separations and image noise levels. With 3 subsequent images from the low resolution (LR) sequence of a 4-time lower spatial resolution and another 2 images from the high resolution (HR) sequence of a 20-time lower frame rate, the proposed approach achieved an average PSNR of 37.57 dB and 35.15 dB, respectively. When coupled with the appropriate combination of high-speed cameras, the proposed approach will enhance the performance and therefore scientific value of the UHS x-ray imaging experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。