用2D追踪点模型实现内窥镜图像中无标记3D组织运动精准追踪。
Tracking Any Point Methods for Markerless 3D Tissue Tracking in Endoscopic Stereo Images
- 结合时间追踪与立体匹配双模型，从双目图像推算3D运动。
- 在鸡组织上实现10mm/s速度下1.1mm的欧氏误差，精度高。
- 适用于微创手术中实时导航与机器人辅助，适合临床场景。
微创手术面临组织动态运动和视野受限的挑战。准确的组织追踪可支持术中导航，提升安全性并实现智能机器人辅助。本文提出一种基于2D Tracking Any Point（TAP）网络的无标记3D组织追踪新方法。通过融合两个CoTracker模型——一个用于时间追踪，一个用于立体匹配——从双目内窥镜图像中估计3D运动。我们在临床腹腔镜系统与模拟组织运动的机械臂上进行评估，实验使用3D打印假体和鸡组织假体。在鸡组织假体上的追踪表现更优，速度达10 mm/s时欧氏距离误差低至1.1 mm。结果表明，基于TAP的模型在复杂手术场景中具备高精度无标记3D追踪潜力。
原文摘要 · Abstract (English)
Minimally invasive surgery presents challenges such as dynamic tissue motion and a limited field of view. Accurate tissue tracking has the potential to support surgical guidance, improve safety by helping avoid damage to sensitive structures, and enable context-aware robotic assistance during complex procedures. In this work, we propose a novel method for markerless 3D tissue tracking by leveraging 2D Tracking Any Point (TAP) networks. Our method combines two CoTracker models, one for temporal tracking and one for stereo matching, to estimate 3D motion from stereo endoscopic images. We evaluate the system using a clinical laparoscopic setup and a robotic arm simulating tissue motion, with experiments conducted on a synthetic 3D-printed phantom and a chicken tissue phantom. Tracking on the chicken tissue phantom yielded more reliable results, with Euclidean distance errors as low as 1.1 mm at a velocity of 10 mm/s. These findings highlight the potential of TAP-based models for accurate, markerless 3D tracking in challenging surgical scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。