arXiv:2603.18544eess.IVcs.AI2026-03

用自由涂鸦交互式精准分割手术图像,支持多次修正。

SCISSR: Scribble-Conditioned Interactive Surgical Segmentation and Refinement

  • 将手绘涂鸦转为密集提示嵌入,实现逐轮修正。
  • 在EndoVis 2018上达95.41% Dice,CholecSeg8k上96.30% Dice。
  • 轻量模块可适配SAM系列,适合医疗影像交互分割。

手术场景中组织与器械的精准分割因形状不规则、结构细长、反光及频繁遮挡而高度依赖标注。尽管SAM模型支持点、框和掩码提示,但点提示过于稀疏,框提示又过于粗糙,难以定位此类挑战目标。我们提出SCISSR,一种支持涂鸦提示的交互式手术场景分割框架。它引入轻量级涂鸦编码器,将自由手绘涂鸦转化为与掩码解码器兼容的密集提示嵌入,通过在错误区域绘制修正笔画实现目标对象的迭代优化。由于所有新增模块(涂鸦编码器、空间门控融合、LoRA适配器)仅通过标准嵌入接口与主干网络交互,该框架不绑定单一模型:本文基于SAM 2构建,但相同组件可无缝迁移至其他提示驱动分割架构(如SAM 3),无需结构修改。为保留预训练能力,仅训练这些轻量新增部分,其余主干保持冻结。在EndoVis 2018上的实验显示强域内性能,而在跨域数据集CholecSeg8k上的评估进一步验证了其泛化能力。SCISSR在EndoVis 2018上经五轮交互达95.41% Dice,在CholecSeg8k上三轮交互达96.30% Dice,优于迭代点提示方法。

原文摘要 · Abstract (English)

Accurate segmentation of tissues and instruments in surgical scenes is annotation-intensive due to irregular shapes, thin structures, specularities, and frequent occlusions. While SAM models support point, box, and mask prompts, points are often too sparse and boxes too coarse to localize such challenging targets. We present SCISSR, a scribble-promptable framework for interactive surgical scene segmentation. It introduces a lightweight Scribble Encoder that converts freehand scribbles into dense prompt embeddings compatible with the mask decoder, enabling iterative refinement for a target object by drawing corrective strokes on error regions. Because all added modules (the Scribble Encoder, Spatial Gated Fusion, and LoRA adapters) interact with the backbone only through its standard embedding interfaces, the framework is not tied to a single model: we build on SAM 2 in this work, yet the same components transfer to other prompt-driven segmentation architectures such as SAM 3 without structural modification. To preserve pre-trained capabilities, we train only these lightweight additions while keeping the remaining backbone frozen. Experiments on EndoVis 2018 demonstrate strong in-domain performance, while evaluation on the out-of-distribution CholecSeg8k further confirms robustness across surgical domains. SCISSR achieves 95.41% Dice on EndoVis 2018 with five interaction rounds and 96.30% Dice on CholecSeg8k with three interaction rounds, outperforming iterative point prompting on both benchmarks.

医学图像交互分割涂鸦提示SAM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。