用手术仿真+扩散模型生成逼真内镜数据,提升智能手术系统训练效果。
SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Models
- 构建全流程手术仿真器自动生成多类标注数据。
- 结合扩散模型转换合成图像,使其视觉特征接近真实内镜画面。
- 适合研究医学图像生成、智能手术辅助的团队使用。
计算机辅助手术(CAS)系统通过为外科医生提供高级支持来提升手术执行效率与结果。这些系统通常依赖于复杂且难以标注的深度学习模型。虽然合成数据生成可缓解这一挑战,但提升数据真实性至关重要。本文提出一种多阶段合成数据生成流程,包含一个功能完整的手术仿真器,能自动生成现代CAS系统所需的所有标注。该仿真器生成的标注种类远超现有公开合成数据集。同时,其对器械与可变形解剖结构之间的交互模拟更复杂、更真实,优于现有方法。为进一步弥合合成数据与真实数据间的视觉差距,我们提出一种基于Stable Diffusion(SD)和低秩适配(LoRA)的轻量级、灵活的图像到图像翻译方法。该方法仅需少量标注数据即可高效训练,并保持仿真器生成的标注完整性。实验验证表明,该管道可将合成图像转化为具有真实世界特征的图像,且具备良好的泛化能力,显著提升训练效果与CAS引导性能。代码与数据集已开源:https://github.com/SanoScience/SimuScope。
原文摘要 · Abstract (English)
Computer-assisted surgical (CAS) systems enhance surgical execution and outcomes by providing advanced support to surgeons. These systems often rely on deep learning models trained on complex, challenging-to-annotate data. While synthetic data generation can address these challenges, enhancing the realism of such data is crucial. This work introduces a multi-stage pipeline for generating realistic synthetic data, featuring a fully-fledged surgical simulator that automatically produces all necessary annotations for modern CAS systems. This simulator generates a wide set of annotations that surpass those available in public synthetic datasets. Additionally, it offers a more complex and realistic simulation of surgical interactions, including the dynamics between surgical instruments and deformable anatomical environments, outperforming existing approaches. To further bridge the visual gap between synthetic and real data, we propose a lightweight and flexible image-to-image translation method based on Stable Diffusion (SD) and Low-Rank Adaptation (LoRA). This method leverages a limited amount of annotated data, enables efficient training, and maintains the integrity of annotations generated by our simulator. The proposed pipeline is experimentally validated and can translate synthetic images into images with real-world characteristics, which can generalize to real-world context, thereby improving both training and CAS guidance. The code and the dataset are available at https://github.com/SanoScience/SimuScope.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。