arXiv:2501.08115cs.CVcs.LG2025-01被引 2

RoHan通过合成手套图像与半监督学习,提升手术室手部检测鲁棒性。

RoHan: Robust Hand Detection in Operation Room

  • 用合成手套图像增强公开数据集,解决无手套标注难题
  • 通过迭代预测与帧筛选,在真实手术场景中实现高精度检测
  • 减少人工标注需求,适合医疗视觉系统快速落地

手部定位在计算机视觉领域受到广泛关注。尽管已有多个包含不同视角和场景的手部标注数据集,但域迁移技术在手术环境中的表现仍不佳,主要由于手套手部实例稀缺及手术室特有的挑战。因此,针对手术室环境的手部检测模型需大量训练和昂贵的标注过程。为克服这些挑战,我们提出「RoHan」——一种针对手术室的鲁棒手部检测新方法,利用先进的半监督域适应技术应对记录条件变化、手套颜色多样性和遮挡等常见问题。该方法包括两个阶段:(1) 数据增强策略,采用「Artificial Gloves」方法,将公开手部数据集通过合成戴手套图像进行扩充;(2) 半监督域适应流程,通过迭代预测优化与高效帧筛选提升真实手术场景下的检测性能。我们在模拟肠吻合修复和大隐静脉移植采集两个数据集上评估该方法,结果显示,RoHan显著减少了对大量标注和训练的需求,为医疗环境中手部检测技术的实际应用铺平道路。

原文摘要 · Abstract (English)

Hand-specific localization has garnered significant interest within the computer vision community. Although there are numerous datasets with hand annotations from various angles and settings, domain transfer techniques frequently struggle in surgical environments. This is mainly due to the limited availability of gloved hand instances and the unique challenges of operating rooms (ORs). Thus, hand-detection models tailored to OR settings require extensive training and expensive annotation processes. To overcome these challenges, we present "RoHan" - a novel approach for robust hand detection in the OR, leveraging advanced semi-supervised domain adaptation techniques to tackle the challenges of varying recording conditions, diverse glove colors, and occlusions common in surgical settings. Our methodology encompasses two main stages: (1) data augmentation strategy that utilizes "Artificial Gloves," a method for augmenting publicly available hand datasets with synthetic images of hands-wearing gloves; (2) semi-supervised domain adaptation pipeline that improves detection performance in real-world OR settings through iterative prediction refinement and efficient frame filtering. We evaluate our method using two datasets: simulated enterotomy repair and saphenous vein graft harvesting. "RoHan" substantially reduces the need for extensive labeling and model training, paving the way for the practical implementation of hand detection technologies in medical settings.

手部检测医学视觉域适应半监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。