arXiv:2608.23254cs.LG2026-08中稿 · ACM Multimedia 202…

用多模态数据建模手术团队互动,生成可解释的改进建议。

From Multimodal Observation to Interpretable Suggestions: Counterfactual Time-Expanded Relational Modeling of Surgical Teams

  • 基于时间扩展图捕捉团队关系与行为随时间演变。
  • 通过反事实分析找出最小有效行为调整以提升团队绩效。
  • 适合关注手术团队协作优化的研究者与临床培训人员。

在手术中,患者安全不仅受技术问题威胁,也受团队协作不佳影响。现有手术AI主要关注视觉流程与技术执行,忽视团队互动建模,错失主动支持临床人员提升协作能力的机会。为此,我们提出一种时序-关系框架,从多模态观测中建模手术团队动态。利用时间扩展图,该方法同时捕捉关系结构与时间演化,在典型低数据的手术场景下仍具鲁棒性。除预测外,该模型还能生成高效、可解释且可操作的建议。具体而言,通过反事实程序识别出与团队绩效提升相关联的个体行为与互动模式的最小但结构化改变。在模拟手术流程的实验中,该方法在多种行为与互动目标上均提升预测性能,并为团队动态提供有意义洞察。本工作将手术AI从结果驱动预测推进至社会基础、团队中心且可行动的新范式,更好理解并支持手术团队技能发展。

原文摘要 · Abstract (English)

In surgery, patient safety is threatened not only by technical issues but also by poor teamwork. However, existing surgical AI-based solutions focus mainly on visual workflow and technical execution, neglecting the modeling of team interactions and missing opportunities to actively support clinicians in improving their teamwork skills. To address this gap, we propose a tempo-relational framework for modeling surgical team dynamics from multimodal observations. By leveraging Time-Expanded graphs, the approach captures both relational structure and temporal evolution, achieving strong expressivity while remaining robust in the low-data regime typical of surgical settings. Beyond prediction, such modeling enables the generation of efficient, interpretable, and actionable suggestions for clinicians. More specifically, we generate suggestions via a counterfactual procedure that identifies minimal yet structured changes in individual behaviors and interaction patterns associated with improvements in team performance. Experiments with simulated surgical procedures show that our approach improves predictive performance in diverse behavioral and interaction goals while offering meaningful insights into team dynamics. This work advances surgical AI beyond outcome-driven prediction towards a socially grounded, team-centric, and actionable paradigm to better understand and support the development of team skills in surgical settings.

团队协作手术AI反事实分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。