用合成数据训练的神经网络，实时追踪手术器械3D姿态
SurgeoNet: Realtime 3D Pose Estimation of Articulated Surgical Instruments from Stereo Images using a Synthetically-trained Network
- 基于立体视觉与合成数据训练的多阶段网络
- 仅靠合成数据即可在真实场景中准确追踪器械
- 适合混合现实手术监控与机器人辅助手术应用
混合现实（MR）环境中的手术监测因对图像决策、技能评估和机器人辅助手术的重要性而受到广泛关注。追踪双手及可动手术器械对这些应用的成功至关重要。由于缺乏标注数据集且任务复杂，相关研究较少。本文提出SurgeoNet，一种基于立体视觉的实时神经网络管道，用于精确检测和跟踪手术器械。该方法受最新神经网络架构（如YOLO和Transformer）启发，仅通过合成数据训练，便在复杂真实场景中展现出良好泛化能力。该方案可轻松扩展至任意新类型可动手术器械。SurgeoNet的代码与数据已公开。
原文摘要 · Abstract (English)
Surgery monitoring in Mixed Reality (MR) environments has recently received substantial focus due to its importance in image-based decisions, skill assessment, and robot-assisted surgery. Tracking hands and articulated surgical instruments is crucial for the success of these applications. Due to the lack of annotated datasets and the complexity of the task, only a few works have addressed this problem. In this work, we present SurgeoNet, a real-time neural network pipeline to accurately detect and track surgical instruments from a stereo VR view. Our multi-stage approach is inspired by state-of-the-art neural-network architectural design, like YOLO and Transformers. We demonstrate the generalization capabilities of SurgeoNet in challenging real-world scenarios, achieved solely through training on synthetic data. The approach can be easily extended to any new set of articulated surgical instruments. SurgeoNet's code and data are publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。