首个由外科协会主导的AI手术质量评估竞赛,聚焦关键安全步骤的自动识别。
The SAGES Critical View of Safety Challenge: A Global Benchmark for AI-Assisted Surgical Quality Assessment
- 构建全球协作数据集,用多专家标注1000段腹腔镜胆囊切除术视频。
- 参赛团队实现评估性能提升17%,校准误差降低80%以上。
- 提供可部署的AI框架与方法论指南,适合临床落地研究者参考。
人工智能在手术质量评估中的进展有望普及专业经验,应用于培训、指导和认证。本研究发起由外科协会组织的首个AI竞赛——SAGES关键安全视图(CVS)挑战赛,以腹腔镜胆囊切除术中的关键安全步骤为范例,该步骤虽普遍推荐但执行不一。来自24个国家54家机构的数百名临床医生与工程师合作,采集并标注了1000段视频,由20位外科专家依据共识验证协议完成标注。挑战赛解决真实临床部署的关键障碍:高精度、主观判断不确定性建模及对临床差异的鲁棒性。为此开发了EndoGlacier框架,用于管理大规模异构手术视频与多标注者工作流。十三支国际团队参与,相较当前最优方法,实现最高17%的性能相对提升,校准误差减少超80%,鲁棒性提高17%。结果分析揭示了影响模型表现的方法趋势,为未来研发可临床部署的鲁棒性AI提供了指导。
原文摘要 · Abstract (English)
Advances in artificial intelligence (AI) for surgical quality assessment promise to democratize access to expertise, with applications in training, guidance, and accreditation. This study presents the SAGES Critical View of Safety (CVS) Challenge, the first AI competition organized by a surgical society, using the CVS in laparoscopic cholecystectomy, a universally recommended yet inconsistently performed safety step, as an exemplar of surgical quality assessment. A global collaboration across 54 institutions in 24 countries engaged hundreds of clinicians and engineers to curate 1,000 videos annotated by 20 surgical experts according to a consensus-validated protocol. The challenge addressed key barriers to real-world deployment in surgery, including achieving high performance, capturing uncertainty in subjective assessment, and ensuring robustness to clinical variability. To enable this scale of effort, we developed EndoGlacier, a framework for managing large, heterogeneous surgical video and multi-annotator workflows. Thirteen international teams participated, achieving up to a 17% relative gain in assessment performance, over 80% reduction in calibration error, and a 17% relative improvement in robustness over the state-of-the-art. Analysis of results highlighted methodological trends linked to model performance, providing guidance for future research toward robust, clinically deployable AI for surgical quality assessment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。