arXiv:2506.21330cs.CVcs.AI2025-06被引 3

用线性时序模型实现全程手术阶段识别,效率远超传统Transformer。

Holistic Surgical Phase Recognition with Hierarchical Input Dependent State Space Models

  • 分层状态空间模型结合局部与全局时序建模,支持长视频处理。
  • 在Cholec80等3个数据集上准确率提升2.8%至12.9%。
  • 适合需要高精度手术流程分析的临床研究与机器人手术系统。

手术流程分析对机器人辅助手术至关重要,但手术时间长导致视频分析困难。现有方法多依赖注意力机制为二次复杂度的Transformer,难以高效处理长视频。本文提出一种分层输入相关状态空间模型,利用状态空间模型的线性扩展特性,在保持全视频决策能力的同时捕捉局部与全局动态。框架包含一个时序一致的视觉特征提取器,通过在特征提取器后添加状态空间模型头来传递时间信息。模型由两个核心模块组成:局部聚合状态空间块用于捕捉精细局部动态,全局关系状态空间块用于建模全视频时序依赖。采用混合离散-连续监督策略,同时传播阶段标签和阶段进展信号。实验表明,该方法在Cholec80、MICCAI2016和Heichole数据集上分别领先当前最优方法2.8%、4.3%和12.9%。代码将在论文录用后公开。

原文摘要 · Abstract (English)

Surgical workflow analysis is essential in robot-assisted surgeries, yet the long duration of such procedures poses significant challenges for comprehensive video analysis. Recent approaches have predominantly relied on transformer models; however, their quadratic attention mechanism restricts efficient processing of lengthy surgical videos. In this paper, we propose a novel hierarchical input-dependent state space model that leverages the linear scaling property of state space models to enable decision making on full-length videos while capturing both local and global dynamics. Our framework incorporates a temporally consistent visual feature extractor, which appends a state space model head to a visual feature extractor to propagate temporal information. The proposed model consists of two key modules: a local-aggregation state space model block that effectively captures intricate local dynamics, and a global-relation state space model block that models temporal dependencies across the entire video. The model is trained using a hybrid discrete-continuous supervision strategy, where both signals of discrete phase labels and continuous phase progresses are propagated through the network. Experiments have shown that our method outperforms the current state-of-the-art methods by a large margin (+2.8% on Cholec80, +4.3% on MICCAI2016, and +12.9% on Heichole datasets). Code will be publicly available after paper acceptance.

手术分析状态空间视频理解机器人手术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。