用单目深度图引导分割，零训练实现精准手术场景识别
See in Depth: Training-Free Surgical Scene Segmentation with Monocular Depth Priors
- 用预训练深度模型生成相对深度图，指导点提示生成掩码
- 在CholecSeg8k上达到35.9% mIoU，比基线高21.2个百分点
- 仅需10%-20%模板仍保持良好性能，适合标注成本高的场景
腹腔镜场景的像素级分割对计算机辅助手术至关重要，但因密集标注成本高昂而难以规模化。本文提出无需训练的深度引导手术场景分割方法（DepSeg），利用单目深度作为几何先验，并结合预训练视觉基础模型。DepSeg首先通过预训练单目深度估计网络生成相对深度图，提出深度引导点提示，由SAM2转化为类无关掩码。每个掩码通过池化预训练视觉特征，再通过模板匹配与标注帧构建的模板库进行分类。在CholecSeg8k数据集上，DepSeg相比直接使用SAM2自动分割的基线（35.9% vs. 14.7% mIoU）显著提升，且仅使用10%–20%物体模板时仍保持竞争力。结果表明，深度引导提示与基于模板的分类可实现高效标注的分割方案。
原文摘要 · Abstract (English)
Pixel-wise segmentation of laparoscopic scenes is essential for computer-assisted surgery but difficult to scale due to the high cost of dense annotations. We propose depth-guided surgical scene segmentation (DepSeg), a training-free framework that utilizes monocular depth as a geometric prior together with pretrained vision foundation models. DepSeg first estimates a relative depth map with a pretrained monocular depth estimation network and proposes depth-guided point prompts, which SAM2 converts into class-agnostic masks. Each mask is then described by a pooled pretrained visual feature and classified via template matching against a template bank built from annotated frames. On the CholecSeg8k dataset, DepSeg improves over a direct SAM2 auto segmentation baseline (35.9% vs. 14.7% mIoU) and maintains competitive performance even when using only 10--20% of the object templates. These results show that depth-guided prompting and template-based classification offer an annotation-efficient segmentation approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。