用分割模型提升手术器械尖端定位精度，解决机器人手术视觉校准难题。
ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detection
- 基于分割掩码输入，端到端学习器械尖端位置。
- 在仿真与真实数据上均优于传统图像处理方法。
- 适合需要高精度器械定位的手术自动化研究者。
在机器人辅助腹腔镜根治性前列腺切除术（RALP）中，器械尖端位置对超声与腹腔镜相机帧的配准至关重要。现有通过达芬奇API获取的尖端位置存在误差，需依赖手动手眼标定。因此，直接基于视觉方法计算相机坐标系下的器械尖端位置成为有吸引力的解决方案。此外，器械尖端检测是手术技能评估与手术自动化的关键环节。然而，由于器械尖端尺寸小且具有关节运动特性，该任务极具挑战性。得益于分割基础模型（如Segment Anything）的发展，器械分割已相对容易。本文探索一种基于深度学习的器械尖端检测方法，以部件级分割掩码为输入。在模拟与真实数据集上的对比实验表明，所提方法优于手工设计的图像处理方法。
原文摘要 · Abstract (English)
In robot-assisted laparoscopic radical prostatectomy (RALP), the location of the instrument tip is important to register the ultrasound frame with the laparoscopic camera frame. A long-standing limitation is that the instrument tip position obtained from the da Vinci API is inaccurate and requires hand-eye calibration. Thus, directly computing the position of the tool tip in the camera frame using the vision-based method becomes an attractive solution. Besides, surgical instrument tip detection is the key component of other tasks, like surgical skill assessment and surgery automation. However, this task is challenging due to the small size of the tool tip and the articulation of the surgical instrument. Surgical instrument segmentation becomes relatively easy due to the emergence of the Segmentation Foundation Model, i.e., Segment Anything. Based on this advancement, we explore the deep learning-based surgical instrument tip detection approach that takes the part-level instrument segmentation mask as input. Comparison experiments with a hand-crafted image-processing approach demonstrate the superiority of the proposed method on simulated and real datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。