用文本和分割图训练本地手术视觉大模型，精准解释肠系膜切除术解剖结构。
Distilling Expert Surgical Knowledge: How to train local surgical VLMs for anatomy explanation in Complete Mesocolic Excision
- 用教师大模型生成无敏感图像的专家标注数据，仅依赖文本和分割掩码。
- 在本地部署的视觉语言模型上微调后，解剖识别准确率显著提升。
- 适合医院内部使用，兼顾隐私保护与手术场景理解能力。
视觉大语言模型（VLM）在辅助诊断中展现出巨大潜力，但在肠系膜完整切除术等专业手术场景中仍难以准确识别与解释解剖标志。为避免患者数据外泄，需部署本地化模型。本文提出一种隐私保护框架，将通用大语言模型的知识蒸馏至轻量级本地VLM。通过仅使用文本上下文和二值分割掩码，无需敏感图像即可生成专家监督数据集，用于局部VLM的监督微调（SFT）与直接偏好优化（DPO）。评估表明，相比基础模型，该方法显著提升了模型在手术领域的知识掌握度。本工作验证了一种高效、合规的数据驱动方式，可训练出面向手术场景、可本地部署的专用视觉语言模型。
原文摘要 · Abstract (English)
Recently, Vision Large Language Models (VLMs) have demonstrated high potential in computer-aided diagnosis and decision-support. However, current VLMs show deficits in domain specific surgical scene understanding, such as identifying and explaining anatomical landmarks during Complete Mesocolic Excision. Additionally, there is a need for locally deployable models to avoid patient data leakage to large VLMs, hosted outside the clinic. We propose a privacy-preserving framework to distill knowledge from large, general-purpose LLMs into an efficient, local VLM. We generate an expert-supervised dataset by prompting a teacher LLM without sensitive images, using only textual context and binary segmentation masks for spatial information. This dataset is used for Supervised Fine-Tuning (SFT) and subsequent Direct Preference Optimization (DPO) of the locally deployable VLM. Our evaluation confirms that finetuning VLMs with our generated datasets increases surgical domain knowledge compared to its base VLM by a large margin. Overall, this work validates a data-efficient and privacy-conforming way to train a surgical domain optimized, locally deployable VLM for surgical scene understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。