用CT和深度图融合,精准估算卧床患者3D姿态与体型。
Multi-modal 3D Pose and Shape Estimation with Computed Tomography
- 结合CT与深度图,通过概率对齐和神经网络联合建模。
- 姿态估计误差降低23%,体型重建准确率提升49.16%。
- 适合手术导航、术后护理等临床高要求场景。
在围术期护理中,精确的卧床患者三维姿态与体型估计(PSE)对于优化术前体位规划、实现医学图像在增强现实手术导航中的精准叠加,以及降低术后长时间不动带来的风险至关重要。传统依赖RGB-D、红外或压力图等模态的方法常因被褥遮挡和复杂体位导致估计不准,影响临床效果。为此,我们提出首个融合常规获取的计算机断层扫描(CT)与深度图的多模态卧床患者3D PSE网络(mPSE-CT)。该模型包含基于概率对应对齐的体型估计模块、改进的神经网络姿态估计模块及最终参数融合模块,能有效重建被遮挡区域,提升3D人体网格估计精度。我们在自研全身体模和志愿者数据集上验证了mPSE-CT,其在姿态与体型估计上分别优于最优基线方法23%和49.16%,展现出在复杂围术期环境中改善临床结果的巨大潜力。
原文摘要 · Abstract (English)
In perioperative care, precise in-bed 3D patient pose and shape estimation (PSE) can be vital in optimizing patient positioning in preoperative planning, enabling accurate overlay of medical images for augmented reality-based surgical navigation, and mitigating risks of prolonged immobility during recovery. Conventional PSE methods relying on modalities such as RGB-D, infrared, or pressure maps often struggle with occlusions caused by bedding and complex patient positioning, leading to inaccurate estimation that can affect clinical outcomes. To address these challenges, we present the first multi-modal in-bed patient 3D PSE network that fuses detailed geometric features extracted from routinely acquired computed tomography (CT) scans with depth maps (mPSE-CT). mPSE-CT incorporates a shape estimation module that utilizes probabilistic correspondence alignment, a pose estimation module with a refined neural network, and a final parameters mixing module. This multi-modal network robustly reconstructs occluded body regions and enhances the accuracy of the estimated 3D human mesh model. We validated mPSE-CT using proprietary whole-body rigid phantom and volunteer datasets in clinical scenarios. mPSE-CT outperformed the best-performing prior method by 23% and 49.16% in pose and shape estimation respectively, demonstrating its potential for improving clinical outcomes in challenging perioperative environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。