arXiv:2608.02188cs.CV2026-08

通过建模器械-组织交互关系,提升手术动作三元组跨中心识别准确率。

SPIRIT: Spatio-temporal Pairwise Relational Modeling of Instrument-Tissue Interactions for Surgical Action Triplet Recognition

论文配图:SPIRIT: Spatio-temporal Pairwise Relational Modeling of Instrument-Tissue Interactions for Surgical Action Triplet Recognition
图 1 · 摘自论文原文
  • 分阶段学习器械、动词、目标的时空特征,再建模三者间成对关系。
  • 在四个中心的数据上达到最优性能,跨中心泛化能力显著提升。
  • 适合需要多中心手术分析与智能辅助的医疗研究者使用。

精细理解手术活动对术中智能辅助(如安全监控、不良事件识别、技能评估)至关重要。手术动作三元组(<器械, 动词, 目标>)提供了器械-组织交互的结构化描述。当前关键挑战是:如何在不同机构间保持三元组表示的可靠性,因手术视频存在采集条件、手术风格、器械使用和组织处理差异。现有数据集也无法支持对中心间迁移能力的显式评估。为此,本文提出SPIRIT框架,通过显式建模器械-组织交互关系,提升三元组表示的跨中心迁移能力。不同于将三元组视为扁平类别,SPIRIT先学习器械、动词、目标的时空特征,再建模其两两关系,并通过多头蒸馏稳定训练过程。为评估该设置,我们构建了MultiBypass-4C-T40数据集,涵盖四个地理分布不同的中心,在罗氏-伊恩胃旁路术中进行密集三元组识别,附带阶段与步骤标注。在多种评估协议下,SPIRIT均优于近期强基线模型,验证了显式关系推理对多中心三元组识别的价值。代码将开源至https://github.com/CAMMA-public/multibypass-4c-t40。

原文摘要 · Abstract (English)

Fine-grained understanding of surgical activity is essential for context-aware assistance in the operating room, including safety monitoring, adverse event identification, and skill assessment. Surgical action triplets, defined as tuples of the form <instrument, verb, target>, provide a structured description of instrument-tissue interactions. A key open problem, however, is how to learn triplet representations that remain reliable across institutions, where surgical video varies in acquisition conditions, surgeon style, tool usage, and tissue handling, while existing triplet datasets do not support explicit evaluation of center-wise transfer. To address this problem, we propose \textbf{SPIRIT}, a structured framework for surgical action triplet recognition designed to learn interaction representations that transfer more reliably across centers. Instead of treating each triplet as a flat class label, SPIRIT first learns spatio-temporal representations for instruments, verbs, and targets, then models their pairwise relations, and finally composes them into coherent triplet predictions, with multi-head distillation used to stabilize learning. To evaluate this setting, we establish \textbf{MultiBypass-4C-T40}, a multi-centric dataset for dense surgical action triplet recognition in Roux-en-Y gastric bypass across four geographically distinct centers, with auxiliary phase and step annotations. Across multiple evaluation protocols, SPIRIT consistently outperforms strong recent baselines, highlighting the value of explicit relational reasoning for multi-centric triplet recognition. Code will be available at https://github.com/CAMMA-public/multibypass-4c-t40.

手术分析三元组识别跨中心迁移关系建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。