用双视角视觉变换器同时分割导管并估计3D受力,提升介入手术精准度。
TransForSeg: A Multitask Stereo ViT for Joint Stereo Segmentation and 3D Force Estimation in Catheterization
- 设计双输入序列的立体视觉变换器,直接捕捉跨视角长程依赖。
- 在合成噪声图像上实现98.2%分割准确率与0.47N力估计算法误差。
- 适合医疗机器人、手术导航系统研发人员参考使用。
近年来,多任务深度学习模型通过端到端架构提升了介入手术的触觉与视觉感知能力。该方法基于分割与力估计头,定位X光图像中的导管并根据其形变估算施加压力。现有立体视觉架构采用基于CNN的编码器-解码器结构,捕捉双视角X光图像间的依赖关系,实现3D力估计与立体分割。本文提出一种新型编码器-解码器视觉变换器,将双视角X光图像作为独立序列处理,利用变换器直接建模长程依赖,无需逐步扩大感受野。编码器与解码器生成的嵌入特征共享至两个分割头,解码器融合信息则用于回归头进行3D力估计。该模型在不同噪声水平的合成X光图像上进行了大量实验,对比了最先进的纯分割模型、基于视觉的导管力估计方法以及多任务导管分割与力估计方法,结果表明其在导管分割与力估计两项任务上均达到新基准性能。
原文摘要 · Abstract (English)
Recently, the emergence of multitask deep learning models has enhanced catheterization procedures by providing tactile and visual perception data through an end-to-end architecture. This information is derived from a segmentation and force estimation head, which localizes the catheter in X-ray images and estimates the applied pressure based on its deflection within the image. These stereo vision architectures incorporate a CNN-based encoder-decoder that captures the dependencies between X-ray images from two viewpoints, enabling simultaneous 3D force estimation and stereo segmentation of the catheter. With these tasks in mind, this work approaches the problem from a new perspective. We propose a novel encoder-decoder Vision Transformer model that processes two input X-ray images as separate sequences. Given sequences of X-ray patches from two perspectives, the transformer captures long-range dependencies without the need to gradually expand the receptive field for either image. The embeddings generated by both the encoder and decoder are fed into two shared segmentation heads, while a regression head employs the fused information from the decoder for 3D force estimation. The proposed model is a stereo Vision Transformer capable of simultaneously segmenting the catheter from two angles while estimating the generated forces at its tip in 3D. This model has undergone extensive experiments on synthetic X-ray images with various noise levels and has been compared against state-of-the-art pure segmentation models, vision-based catheter force estimation methods, and a multitask catheter segmentation and force estimation approach. It outperforms existing models, setting a new state-of-the-art in both catheter segmentation and force estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。