用生成模型预测手术全流程,提前30分钟预判下一步操作。
SWAG: Long-term Surgical Workflow Prediction with Generative-based Anticipation

- 基于生成式框架,融合相位识别与未来步骤预测
- 在Cholec80和AutoLaparo21上实现41.3%的长时预测F1分数
- 支持多时序跨度预测,适合临床实时辅助决策
现有方法虽能准确识别当前手术阶段,但缺乏对后续步骤的远期预见能力。当前预测方法通常仅限于短时、单一事件,无法捕捉手术流程中密集、重复且长序列的特性。为此,本文提出SWAG(Surgical Workflow Anticipative Generation)框架,结合生成式方法实现相位识别与前瞻预测。研究对比了单次解码(SP)与自回归解码(AR)两种策略,以分钟级粒度在长时域内生成未来手术阶段序列。提出利用类别转移概率构建新型嵌入,提升预测准确性;并设计剩余时间回归到分类(R2C)的生成框架。在公开数据集Cholec80与AutoLaparo21上评估,使用类别转移概率嵌入的单次解码模型(SP*)在20分钟和30分钟预测窗口下分别达到32.1%和41.3%的F1分数。同时,在阶段剩余时间回归任务中表现优异,2分钟与3分钟预测窗口下的加权平均绝对误差分别为0.32和0.48分钟。该方法在生成解码框架与分类/回归任务间具高度泛化性,实现了手术流程识别与前瞻之间的时序连贯性,为术中流程生成与前瞻性指导提供新路径。
原文摘要 · Abstract (English)
While existing approaches excel at recognising current surgical phases, they provide limited foresight and intraoperative guidance into future procedural steps. Similarly, current anticipation methods are constrained to predicting short-term and single events, neglecting the dense, repetitive, and long sequential nature of surgical workflows. To address these needs and limitations, we propose SWAG (Surgical Workflow Anticipative Generation), a framework that combines phase recognition and anticipation using a generative approach. This paper investigates two distinct decoding methods - single-pass (SP) and auto-regressive (AR) - to generate sequences of future surgical phases at minute intervals over long horizons. We propose a novel embedding approach using class transition probabilities to enhance the accuracy of phase anticipation. Additionally, we propose a generative framework using remaining time regression to classification (R2C). SWAG was evaluated on two publicly available datasets, Cholec80 and AutoLaparo21. Our single-pass model with class transition probability embeddings (SP*) achieves 32.1% and 41.3% F1 scores over 20 and 30 minutes on Cholec80 and AutoLaparo21, respectively. Moreover, our approach competes with existing methods on phase remaining time regression, achieving weighted mean absolute errors of 0.32 and 0.48 minutes for 2- and 3-minute horizons. SWAG demonstrates versatility across generative decoding frame works and classification and regression tasks to create temporal continuity between surgical workflow recognition and anticipation. Our method provides steps towards intraoperative surgical workflow generation for anticipation. Project: https://maxboels.com/research/swag.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。