arXiv:2409.07564eess.IVcs.CV2024-09中稿 · the 27th Internati…被引 3

用心脏影像和临床数据混合预测肺动脉压,无创更安全。

TabMixer: Noninvasive Estimation of the Mean Pulmonary Artery Pressure via Imaging and Tabular Data Mixing

  • 设计TabMixer模块,融合影像与表格数据进行特征交互。
  • 在多个模型上提升肺动脉压预测准确率,优于单一模态方法。
  • 适合心血管疾病研究者和医学AI开发者参考使用。

右心导管检查是诊断肺动脉高压的金标准,通过测量平均肺动脉压(mPAP)实现,但其具有侵入性、成本高、耗时且存在风险。本文首次探索从非侵入性心脏磁共振成像视频中估计mPAP。为增强深度学习模型的预测能力,引入了人口统计学特征与临床测量作为额外模态。受全连接网络架构启发,提出TabMixer新模块,通过空间、时间与通道混合实现影像与表格数据的深度融合。特别地,这是首个利用多层感知机在视觉模型中交换表格信息与影像特征的方法。在mPAP估计任务中验证了TabMixer的有效性,显著提升了卷积神经网络、3D-MLP与视觉变换器的性能,且与现有跨模态模块相当。该方法有望改善同时涉及影像与表格数据的临床流程,尤其在无创mPAP评估方面,可显著提升肺动脉高压患者生活质量。源代码已开源:https://github.com/SanoScience/TabMixer。

原文摘要 · Abstract (English)

Right Heart Catheterization is a gold standard procedure for diagnosing Pulmonary Hypertension by measuring mean Pulmonary Artery Pressure (mPAP). It is invasive, costly, time-consuming and carries risks. In this paper, for the first time, we explore the estimation of mPAP from videos of noninvasive Cardiac Magnetic Resonance Imaging. To enhance the predictive capabilities of Deep Learning models used for this task, we introduce an additional modality in the form of demographic features and clinical measurements. Inspired by all-Multilayer Perceptron architectures, we present TabMixer, a novel module enabling the integration of imaging and tabular data through spatial, temporal and channel mixing. Specifically, we present the first approach that utilizes Multilayer Perceptrons to interchange tabular information with imaging features in vision models. We test TabMixer for mPAP estimation and show that it enhances the performance of Convolutional Neural Networks, 3D-MLP and Vision Transformers while being competitive with previous modules for imaging and tabular data. Our approach has the potential to improve clinical processes involving both modalities, particularly in noninvasive mPAP estimation, thus, significantly enhancing the quality of life for individuals affected by Pulmonary Hypertension. We provide a source code for using TabMixer at https://github.com/SanoScience/TabMixer.

肺动脉压多模态融合医学影像深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。