用虚拟现实+大模型实现手术团队非技术能力的沉浸式训练与自动评估
Integrating Virtual Reality and Large Language Models for Team-Based Non-Technical Skills Training and Evaluation in the Operating Room
- 构建多用户VR平台，结合大模型分析手术团队对话行为
- 在两种腹腔镜急症场景中验证，能准确还原医护角色沟通层级
- 适合外科培训、医疗教育研究者及医学模拟中心使用
尽管有效协作与沟通对手术安全至关重要，但非技术技能（NTS）的系统化训练仍远落后于技术模拟。美国外科医师学会/高级外科医生培训项目（ACS/APDS）第三阶段课程要求可扩展的工具，以在腹腔镜急症中同时开展教学与客观评估。我们提出虚拟手术室团队体验（VORTeX），一个融合沉浸式团队模拟与大语言模型（LLM）分析的多用户虚拟现实（VR）平台，用于训练和评估沟通、决策、团队合作与领导力。通过基于外科非技术技能（NOTSS）框架的结构化提示，对团队对话进行分析，实现行为自动分类并生成指向性交互图，量化沟通结构与层级。设计了气胸与腹内出血两个腹腔镜急症场景，以激发真实压力与协作。12名外科专业人员在2024年SAGES会议上完成试点，评价其直观、沉浸且对提升团队协作与沟通有价值。大模型持续生成可解释的沟通网络，显示主刀医生为整合核心，护士为发起者，麻醉师为平衡中介。通过将沉浸式VR与大模型驱动的行为分析结合，VORTeX提供了一种可扩展、符合隐私规范的客观评估框架，支持跨分布式环境的自动化、数据驱动式复盘。
原文摘要 · Abstract (English)
Although effective teamwork and communication are critical to surgical safety, structured training for non-technical skills (NTS) remains limited compared with technical simulation. The ACS/APDS Phase III Team-Based Skills Curriculum calls for scalable tools that both teach and objectively assess these competencies during laparoscopic emergencies. We introduce the Virtual Operating Room Team Experience (VORTeX), a multi-user virtual reality (VR) platform that integrates immersive team simulation with large language model (LLM) analytics to train and evaluate communication, decision-making, teamwork, and leadership. Team dialogue is analyzed using structured prompts derived from the Non-Technical Skills for Surgeons (NOTSS) framework, enabling automated classification of behaviors and generation of directed interaction graphs that quantify communication structure and hierarchy. Two laparoscopic emergency scenarios, pneumothorax and intra-abdominal bleeding, were implemented to elicit realistic stress and collaboration. Twelve surgical professionals completed pilot sessions at the 2024 SAGES conference, rating VORTeX as intuitive, immersive, and valuable for developing teamwork and communication. The LLM consistently produced interpretable communication networks reflecting expected operative hierarchies, with surgeons as central integrators, nurses as initiators, and anesthesiologists as balanced intermediaries. By integrating immersive VR with LLM-driven behavioral analytics, VORTeX provides a scalable, privacy-compliant framework for objective assessment and automated, data-informed debriefing across distributed training environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。