提出TADF框架，实时重建手术中软组织的3D形变。
Tracking-Aware Deformation Field Estimation for Non-rigid 3D Reconstruction in Robotic Surgeries
- 用视觉模型追踪组织关键点，生成2D形变场
- 融合神经隐式网络，在3D空间还原形变，精度更高
- 适合机器人手术中实时形变感知，提升安全性
微创手术因机器人腹腔镜技术迅速发展，极大提升了手术精准度与患者安全性。然而，器械与组织交互时的微小组织形变仍需精确感知，尤其在三维空间中。现有方法多依赖NeRF从不同视角渲染2D视频以消除遮挡，但难以稳定准确预测3D形状及形变。本文提出跟踪感知的3D形变场（TADF）框架，可同步重建3D网格与组织形变。首先通过基础视觉模型追踪软组织关键点，获得高精度2D形变场；随后将该形变场平滑融合至神经隐式重建网络，实现3D空间中的形变估计。实验表明，在两个公开数据集上，本方法相比其他3D神经重建方法，形变估计更准确。
原文摘要 · Abstract (English)
Minimally invasive procedures have been advanced rapidly by the robotic laparoscopic surgery. The latter greatly assists surgeons in sophisticated and precise operations with reduced invasiveness. Nevertheless, it is still safety critical to be aware of even the least tissue deformation during instrument-tissue interactions, especially in 3D space. To address this, recent works rely on NeRF to render 2D videos from different perspectives and eliminate occlusions. However, most of the methods fail to predict the accurate 3D shapes and associated deformation estimates robustly. Differently, we propose Tracking-Aware Deformation Field (TADF), a novel framework which reconstructs the 3D mesh along with the 3D tissue deformation simultaneously. It first tracks the key points of soft tissue by a foundation vision model, providing an accurate 2D deformation field. Then, the 2D deformation field is smoothly incorporated with a neural implicit reconstruction network to obtain tissue deformation in the 3D space. Finally, we experimentally demonstrate that the proposed method provides more accurate deformation estimation compared with other 3D neural reconstruction methods in two public datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。