arXiv:2503.22647eess.IV2025-03被引 1

用模拟手术视频训练模型,实现真实手术中实时错误预测。

Deep learning-enabled prediction of surgical errors during cataract surgery: from simulation to real-world application

  • 用模拟器视频训练深度学习模型,仅需1秒历史数据即可实时预测
  • 在真实手术中经域适应后AUC达0.663,优于直接应用的0.578
  • 无需真实手术标注数据,适合临床辅助与远程指导场景

实时从白内障手术视频中预测技术性错误对远程指导具有重要意义。但手术错误罕见,难以用AI检测。为此,我们利用EyeSi模拟器的视频数据训练错误预测模型,并通过无监督域适应将知识迁移到真实手术场景,无需依赖稀缺的真实手术标注数据。通过对比模拟与真实手术视频片段并预训练模型,实现了快速推理。在1秒预测窗口下,使用600×600像素图像时,模拟器上整体AUC达0.820;使用299×299像素图像时为0.784。在真实场景中,经域适应后最高AUC达0.663,显著优于未适配的0.578。本工作首次仅基于模拟视频实现手术错误预测并成功迁移至真实世界。

原文摘要 · Abstract (English)

Real-time prediction of technical errors from cataract surgical videos can be highly beneficial, particularly for telementoring, which involves remote guidance and mentoring through digital platforms. However, the rarity of surgical errors makes their detection and analysis challenging using artificial intelligence. To tackle this issue, we leveraged videos from the EyeSi Surgical cataract surgery simulator to learn to predict errors and transfer the acquired knowledge to real-world surgical contexts. By employing deep learning models, we demonstrated the feasibility of making real-time predictions using simulator data with a very short temporal history, enabling on-the-fly computations. We then transferred these insights to real-world settings through unsupervised domain adaptation, without relying on labeled videos from real surgeries for training, which are limited. This was achieved by aligning video clips from the simulator with real-world footage and pre-training the models using pretext tasks on both simulated and real surgical data. For a 1-second prediction window on the simulator, we achieved an overall AUC of 0.820 for error prediction using 600$\times$600 pixel images, and 0.784 using smaller 299$\times$299 pixel images. In real-world settings, we obtained an AUC of up to 0.663 with domain adaptation, marking an improvement over direct model application without adaptation, which yielded an AUC of 0.578. To our knowledge, this is the first work to address the tasks of learning surgical error prediction on a simulator using video data only and transferring this knowledge to real-world cataract surgery.

手术预测域适应深度学习白内障手术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。