用低秩适配器高效微调SAM3，实现手术图像精准分割
Parameter-Efficient Adaptation of SAM3 for Prompt-Driven Surgical Concept Segmentation
- 在提示编码器、检测器和追踪器中注入低秩适配器，冻结视觉主干
- 仅优化0.98%参数，单张消费级显卡即可训练
- 显著优于零样本SAM3，可直接用于机器人手术重建与仿真
高效手术分割有助于临床诊断、术中监测及下游机器人重建与仿真流程。尽管提示驱动的基础模型如分割一切模型3（SAM3）在自然图像上表现优异，但其预训练数据与手术数据存在领域差异，导致分割精度下降。此外，现有医疗SAM方法需全参数微调，计算开销大、效率低。为此，本文提出对SAM3进行参数高效的低秩适配（LoRA）微调，用于提示驱动的手术概念分割。通过在提示编码器、检测器和追踪器中注入低秩适配器，同时完全冻结视觉主干，仅优化总参数的0.98%，支持在单张消费级GPU上训练。全面实验表明，该方法持续优于零样本SAM3及其他主流基线，生成的分割结果可直接部署至下游机器人手术场景重建与物理仿真流程。
原文摘要 · Abstract (English)
Efficient surgical segmentation empowers clinical diagnosis, intraoperative monitoring, and downstream robotic pipelines for reconstruction and simulation. Although prompt-driven foundation models like Segment Anything Model 3 (SAM3) achieve strong segmentation performance on natural images, surgical data exhibits domain gaps against its pre-training data, resulting in degraded segmentation accuracy. Furthermore, existing medical SAM methods require full-parameter fine-tuning, incurring heavy computational consumption and low efficiency. To address these limitations, this work proposes a parameter-efficient Low-Rank Adaptation (LoRA) adaptation of SAM3 for surgical concept segmentation. We inject low-rank adapters into the prompt encoder, detector and tracker while fully freezing the vision backbone, which only optimizes 0.98% of the total model parameters and supports training on a single consumer GPU. Comprehensive experiments demonstrate that our method consistently outperforms zero-shot SAM3 and other mainstream baselines, and the generated segmentation results can be directly deployed to support downstream robotic surgical scene reconstruction and physical simulation pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。