arXiv:2606.04992cs.CVcs.HC2026-06

无需标记的多摄像头AR系统,帮护士更高效地组装手术器械。

Multi-Camera AR Guidance System for Surgical Instrument Handling and Assembly: Investigating Workload and Efficiency

论文配图:Multi-Camera AR Guidance System for Surgical Instrument Handling and Assembly: Investigating Workload and Efficiency
图 1 · 摘自论文原文
  • 用多摄像头和合成数据训练的6D姿态估计,实现无标记实时定位。
  • 实测任务时间减少21.3%(4.76分钟),工作负荷显著降低。
  • 特别适合不熟悉器械的护士,提升操作清晰度与独立性。

手术中器械的处理与组装对刷手护士带来高认知负荷,尤其面对陌生器械时。本文提出一种结合多摄像头6D姿态估计与头戴式显示设备的增强现实(AR)在位引导系统,无需额外标记。通过已知物体实现姿态估计与连续相机标定,6D姿态估计网络仅基于合成数据训练,以提升泛化能力与实际适用性。系统通过提示气泡定位与分步组装动画提供引导,用户可通过注视选择与脚踏开关切换步骤。技术评估显示本方法优于现有先进姿态估计算法。在膝关节置换手术模拟中,29名刷手护士参与对照实验,相比纸质手册,AR引导使任务完成时间减少21.3%(4.76分钟),主观负荷显著下降。经验较少的护士受益更明显,错误频率两组相当。定性反馈表明流程更清晰、信息过载减少、感知独立性提高。综上,该无标记多摄像头AR引导系统可客观与主观地提升术中器械操作表现,尤其适用于缺乏经验的护士。

原文摘要 · Abstract (English)

The handling and assembly of instruments during surgery imposes high cognitive demands on scrub nurses, particularly when instruments are unfamiliar. We present a supporting guidance system for surgical instrumentation that combines multi-camera 6D pose estimation with augmented reality in-situ visualization on a head-mounted display without the requirement for additional markers. Pose estimation and consecutive camera calibration are achieved through known objects. The 6D pose estimation network is trained purely on synthetic data, aiming for better generalizability and real-world applicability. The AR guidance displays tooltip localization cues and step-wise assembly animations. Via gaze-based selection and a foot pedal, users can switch between assembly steps in intraoperative use. In a technical evaluation, our approach outperforms state-of-art 6D pose estimation. A user study with 29 scrub nurses was conducted in a surgical simulation of knee arthroplasty, comparing the system against a paper manual. AR guidance significantly reduced the perceived workload compared. Objectively, AR guidance reduced task completion time by 21.3\% (4.76 minutes). Specifically, scrub nurses less experienced with the instrument set benefited when using the system. Error frequencies were comparable between conditions. Qualitative feedback highlighted improved process clarity, reduced information overload, and perceived independence. To summarize, our marker-free multi-camera AR guidance approach for surgical instruments can, subjectively and objectively, improve intraoperative instrumentation performance, particularly for untrained scrub nurses.

AR导航手术辅助多摄像头姿态估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。