arXiv:2604.07577cs.CV2026-04

用视觉模型自动识别手术器械交接,提升手术室安全与效率

Event-Level Detection of Surgical Instrument Handovers in Videos with Interpretable Vision Models

  • 结合ViT与LSTM的时空框架,联合检测交接事件与方向
  • 在肾移植视频上实现0.84的交接检测F1,方向分类均值达0.72
  • 通过注意力可视化揭示关键交互区域,增强模型可解释性

可靠的手术器械交接监测对保障术中效率与患者安全至关重要。由于频繁遮挡、背景干扰及交互过程的时间动态性,术中视频的自动交接检测仍具挑战。本文提出一种时空视觉框架,用于手术视频中器械交接的事件级检测与方向分类。模型采用视觉变换器(ViT)提取空间特征,结合单向长短期记忆网络(LSTM)进行时间聚合,并以统一多任务形式同时预测交接发生与交互方向,避免级联流水线中的误差传播。预测置信度构成视频时间信号,通过峰值检测定位离散交接事件。在肾移植手术数据集上的实验表明,该方法交接检测F1得分为0.84,方向分类平均F1为0.72,优于单任务变体及VideoMamba基线,且检测性能相当。为提升可解释性,采用Layer-CAM注意力图可视化模型决策依据,突出手-器械交互关键区域。

原文摘要 · Abstract (English)

Reliable monitoring of surgical instrument exchanges is essential for maintaining procedural efficiency and patient safety in the operating room. Automatic detection of instrument handovers in intraoperative video remains challenging due to frequent occlusions, background clutter, and the temporally evolving nature of interaction events. We propose a spatiotemporal vision framework for event-level detection and direction classification of surgical instrument handovers in surgical videos. The model combines a Vision Transformer (ViT) backbone for spatial feature extraction with a unidirectional Long Short-Term Memory (LSTM) network for temporal aggregation. A unified multi-task formulation jointly predicts handover occurrence and interaction direction, enabling consistent modeling of transfer dynamics while avoiding error propagation typical of cascaded pipelines. Predicted confidence scores form a temporal signal over the video, from which discrete handover events are identified via peak detection. Experiments on a dataset of kidney transplant procedures demonstrate strong performance, achieving an F1-score of 0.84 for handover detection and a mean F1-score of 0.72 for direction classification, outperforming both a single-task variant and a VideoMamba-based baseline for direction prediction while maintaining comparable detection performance. To improve interpretability, we employ Layer-CAM attribution to visualize spatial regions driving model decisions, highlighting hand-instrument interaction cues.

手术分析视觉模型可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。