无需相机位姿,实时恢复腹腔镜下组织形变,精度达0.37mm
Online camera-pose-free stereo endoscopic tissue deformation recovery with tissue-invariant vision-biomechanics consistency
- 用点与导数地图建模组织几何,6参数描述刚体运动+3参数描述局部形变
- 在遮挡或移出视野时仍稳定重建,非遮挡区误差0.37±0.27mm,遮挡区0.39±0.21mm
- 适合手术导航与软组织自主操作,可输出表面应变分布供力学分析
基于双目腹腔镜图像的组织形变恢复对工具-组织交互分析至关重要,有助于手术导航与自主软组织操作。以往研究受限于相机运动、遮挡、大形变、缺乏组织特异性生物力学先验及依赖离线处理。本文将组织几何表示为3D点与导数图,形变表示为3D位移与局部形变图,单个表面点使用6个参数描述刚体运动,3个参数描述局部形变。方法在相机中心设定下建模,所有运动视为相对于相机的场景运动,通过优化帧间形变实现帧对齐,无需估计相机位姿。引入规范映射概念,实现在线优化。在活体与离体腹腔镜数据集上进行定量与定性实验。输入深度与光流后,即使组织部分遮挡或移出视野,仍能稳定建模几何与形变。非遮挡区域表面距离误差为0.37±0.27 mm,遮挡区域为0.39±0.21 mm。方法还可额外输出不同操作下的表面应变分布,作为力学分析的辅助模态。
原文摘要 · Abstract (English)
Tissue deformation recovery based on stereo endoscopic images is crucial for tool-tissue interaction analysis and benefits surgical navigation and autonomous soft tissue manipulation. Previous research suffers from the problems raised from camera motion, occlusion, large tissue deformation, lack of tissue-specific biomechanical priors, and reliance on offline processing. Unlike previous studies where the tissue geometry and deformation are represented by 3D points and displacements, the proposed method models tissue geometry as the 3D point and derivative map and tissue deformation as the 3D displacement and local deformation map. For a single surface point, 6 parameters are used to describe its rigid motion and 3 parameters for its local deformation. The method is formulated under the camera-centric setting, where all motions are regarded as the scene motion with respect to the camera. Inter-frame alignment is realized by optimizing the inter-frame deformation, making it unnecessary to estimate camera pose. The concept of the canonical map is introduced to optimize tissue geometry and deformation in an online approach. Quantitative and qualitative experiments were conducted using in vivo and ex vivo laparoscopic datasets. With the inputs of depth and optical flow, the method stably models tissue geometry and deformation even when the tissue is partially occluded or moving outside the field of view. Results show that the 3D reconstruction accuracy in the non-occluded and occluded areas reaches 0.37$\pm$0.27 mm and 0.39$\pm$0.21 mm in terms of surface distance, respectively. The method can also estimate surface strain distribution during various manipulations as an extra modality for mechanical-based analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。