首次标注手术室团队沟通时间,用视频分析自动识别关键对话时段。
When do they StOP?: A First Step Towards Automatically Identifying Team Communication in the Operating Room
- 通过多视角视频捕捉并标注团队活动时间点
- 在100+小时数据上实现比现有方法更优的定位精度
- 适合研究手术流程分析与智能辅助系统开发人员
手术表现不仅依赖外科医生的技术能力,还受术中各专业团队间沟通的影响。因此,自动识别手术室内的团队沟通对患者安全及计算机辅助手术流程分析、术中支持系统的发展至关重要。本文首次提出检测包含全体成员参与的沟通环节——即团队术前核查(Time-out)和停机协议(StOP?)——的任务,目标是定位其在手术视频中的起止时间。我们构建了一个真实手术数据集Team-OR,包含超过一百小时的多视角手术视频,涵盖33次术前核查和22次停机协议活动的时间标注。在此基础上,提出一种新的群体活动检测方法,融合场景上下文与动作特征,采用高效神经网络模型输出结果。实验表明,该方法在Team-OR数据集上优于现有的最先进时序动作检测方法,同时揭示了当前手术室内群体活动研究的不足,凸显本数据集的重要价值。
原文摘要 · Abstract (English)
Purpose: Surgical performance depends not only on surgeons' technical skills but also on team communication within and across the different professional groups present during the operation. Therefore, automatically identifying team communication in the OR is crucial for patient safety and advances in the development of computer-assisted surgical workflow analysis and intra-operative support systems. To take the first step, we propose a new task of detecting communication briefings involving all OR team members, i.e. the team Time-out and the StOP?-protocol, by localizing their start and end times in video recordings of surgical operations. Methods: We generate an OR dataset of real surgeries, called Team-OR, with more than one hundred hours of surgical videos captured by the multi-view camera system in the OR. The dataset contains temporal annotations of 33 Time-out and 22 StOP?-protocol activities in total. We then propose a novel group activity detection approach, where we encode both scene context and action features, and use an efficient neural network model to output the results. Results: The experimental results on the Team-OR dataset show that our approach outperforms existing state-of-the-art temporal action detection approaches. It also demonstrates the lack of research on group activities in the OR, proving the significance of our dataset. Conclusion: We investigate the Team Time-Out and the StOP?-protocol in the OR, by presenting the first OR dataset with temporal annotations of group activities protocols, and introducing a novel group activity detection approach that outperforms existing approaches. Code is available at https://github.com/CAMMA-public/Team-OR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。