自动识别达芬奇手术系统中摄像头开启状态，助力手术数据分析
XiCAD: Camera Activation Detection in the Da Vinci Xi User Interface
- 用ResNet18轻量模型定位界面摄像头位置及激活状态
- 在超7万帧数据上实现0.993~1.000的F1分数，零误检
- 适合做手术视频分析、工具追踪与自动化控制的研究者
机器人辅助微创手术依赖内窥镜视频作为唯一术中视觉反馈。达芬奇Xi系统在界面上叠加图形化用户界面（UI），显示各机械臂状态，包括摄像头机械臂的激活状态。检测该激活状态可提供宝贵元数据，如摄像头移动信息，支持下游手术数据科学任务，包括器械追踪、技能评估或摄像头控制自动化。我们开发了一个基于ResNet18卷积神经网络的轻量级管道，可自动识别达芬奇Xi UI中摄像头图块的位置及其激活状态。模型在手工标注的SurgToolLoc数据集上进行微调，并在三个公开数据集上进行评估，涵盖超过70,000帧。结果表明，模型在二值化摄像头激活检测中实现了0.993至1.000的F1分数，所有情况下均正确定位摄像头图块，且无多重摄像头误检。所提管道可实现手术视频中摄像头激活元数据的可靠、实时提取，促进多种下游应用的自动化预处理与分析。所有代码、训练模型及标注数据均已公开。
原文摘要 · Abstract (English)
Purpose: Robot-assisted minimally invasive surgery relies on endoscopic video as the sole intraoperative visual feedback. The DaVinci Xi system overlays a graphical user interface (UI) that indicates the state of each robotic arm, including the activation of the endoscope arm. Detecting this activation provides valuable metadata such as camera movement information, which can support downstream surgical data science tasks including tool tracking, skill assessment, or camera control automation. Methods: We developed a lightweight pipeline based on a ResNet18 convolutional neural network to automatically identify the position of the camera tile and its activation state within the DaVinci Xi UI. The model was fine-tuned on manually annotated data from the SurgToolLoc dataset and evaluated across three public datasets comprising over 70,000 frames. Results: The model achieved F1-scores between 0.993 and 1.000 for the binary detection of active cameras and correctly localized the camera tile in all cases without false multiple-camera detections. Conclusion: The proposed pipeline enables reliable, real-time extraction of camera activation metadata from surgical videos, facilitating automated preprocessing and analysis for diverse downstream applications. All code, trained models, and annotations are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。