提出统一模型SurgPLAN++,兼顾手术阶段在线与离线识别。
SurgPLAN++: Universal Surgical Phase Localization Network for Online and Offline Inference
- 用时序检测思路预测整段手术阶段,而非逐帧分类。
- 在线模式下通过镜像等增强生成伪完整视频提升精度。
- 离线模式可迭代优化预测,适合临床回顾性分析。
手术阶段识别对理解手术视频至关重要。现有研究多聚焦在线识别,依赖前序帧预测当前帧,但将任务建模为逐帧分类,缺乏全过程上下文,导致预测不连贯。此外,离线识别在临床回顾中需求迫切,但现有在线方法未充分分析整段视频,限制了准确性。为此,我们提出通用的手术阶段定位网络SurgPLAN++,基于时序检测原则。为实现全局理解,SurgPLAN++采用阶段定位策略,通过阶段提议预测全视频阶段片段。在线分析中,通过镜像、中心复制和下采样将流式视频扩展为伪完整视频,生成高质量阶段提议。离线分析中,利用全局预测框架在每次在线推理中持续优化先前预测,显著提升识别精度。大量实验验证其有效性,SurgPLAN++在在线与离线模式下均超越现有最优方法。源码见https://github.com/franciszchen/SurgPLAN-Plus。
原文摘要 · Abstract (English)
Surgical phase recognition is critical for assisting surgeons in understanding surgical videos. Existing studies focused more on online surgical phase recognition, by leveraging preceding frames to predict the current frame. Despite great progress, they formulated the task as a series of frame-wise classification, which resulted in a lack of global context of the entire procedure and incoherent predictions. Moreover, besides online analysis, accurate offline surgical phase recognition is also in significant clinical need for retrospective analysis, and existing online algorithms do not fully analyze the entire video, thereby limiting accuracy in offline analysis. To overcome these challenges and enhance both online and offline inference capabilities, we propose a universal Surgical Phase Localization Network, named SurgPLAN++, with the principle of temporal detection. To ensure a global understanding of the surgical procedure, we devise a phase localization strategy for SurgPLAN++ to predict phase segments across the entire video through phase proposals. For online analysis, to generate high-quality phase proposals, SurgPLAN++ incorporates a data augmentation strategy to extend the streaming video into a pseudo-complete video through mirroring, center-duplication, and down-sampling. For offline analysis, SurgPLAN++ capitalizes on its global phase prediction framework to continuously refine preceding predictions during each online inference step, thereby significantly improving the accuracy of phase recognition. We perform extensive experiments to validate the effectiveness, and our SurgPLAN++ achieves remarkable performance in both online and offline modes, which outperforms state-of-the-art methods. The source code is available at https://github.com/franciszchen/SurgPLAN-Plus.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。