arXiv:2601.12012cs.RO2026-01被引 1

用多模态数据实时评估机器人手术技能,提升训练反馈精准度。

Model selection and real-time skill assessment for suturing in robotic surgery

  • 融合运动与视觉数据的深度学习模型实现高精度实时预测。
  • 专家级操作数据训练的模型泛化能力更强,准确率显著提升。
  • 适用于手术培训系统,尤其适合需要精细技能评估的场景。

自动化反馈系统有望为机器人辅助手术的训练与评估提供客观依据。本研究基于达芬奇手术系统采集的数据,开展三项分析:模型设计、实时性能表现及基于技能水平的交叉验证训练。针对模型设计,评估了融合运动与视觉数据的多模态深度学习模型在预测手术技能水平上的效果,采用均值斯皮尔曼等级相关系数进行评估,结果表明融合模型在实时预测中持续优于单模态基线模型。在实时性能方面,观察到预测趋势随术者动作变化而动态响应。在技能水平交叉验证中,分别以不同技能水平的术者数据训练模型,结果显示基于高技能示范数据训练的模型性能更优,且对同水平参与者具有良好的泛化能力。研究证实,多模态学习可实现更稳定的细粒度手术表现评估,并强调了专家级训练数据对模型泛化的重要性。

原文摘要 · Abstract (English)

Automated feedback systems have the potential to provide objective skill assessment for training and evaluation in robot-assisted surgery. In this study, we examine methods to achieve real-time prediction of surgical skill level in real-time based on Objective Structured Assessment of Technical Skills (OSATS) scores. Using data acquired from the da Vinci Surgical System, we carry out three main analyses, focusing on model design, their real-time performance, and their skill-level-based cross-validation training. For the model design, we evaluate the effectiveness of multimodal deep learning models for predicting surgical skill levels using synchronized kinematic and vision data. Our models include separate unimodal baselines and fusion architectures that integrate features from both modalities and are evaluated using mean Spearman's correlation coefficients, demonstrating that the fusion model consistently outperforms unimodal models for real-time predictions. For the real-time performance, we observe the prediction's trend over time and highlight correlation with the surgeon's gestures. For the skill-level-based cross-validation, we separately trained models on surgeons with different skill levels, which showed that high-skill demonstrations allow for better performance than those trained on low-skilled ones and generalize well to similarly skilled participants. Our findings show that multimodal learning allows more stable fine-grained evaluation of surgical performance and highlights the value of expert-level training data for model generalization.

手术评估多模态学习机器人手术

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。