arXiv:2605.17638cs.CV2026-05

通过多视角视觉重建手术中医护人员与物体的接触时空轨迹。

TouchMap-OR: Multi-View 3D Mapping of Hand-Surface Contacts

论文配图:TouchMap-OR: Multi-View 3D Mapping of Hand-Surface Contacts
图 1 · 摘自论文原文
  • 融合多视角RGB-D数据,重建带身份标识的3D手部与人体骨骼轨迹。
  • 接触事件检测F1达0.75,身份识别准确率96%。
  • 适合医疗感染控制研究者与手术流程分析人员。

临床医护人员、患者与医疗设备之间的手-表面交互在医疗操作中是病原体传播的核心环节,但这类交互目前仍难以观测,因现有防控措施依赖人工记录且无法重建详细的接触历史。本文提出触碰重建问题的建模框架,并引入多视角RGB-D视觉系统TouchMap-OR,该系统可建模医护人员、可动手部几何结构及临床环境的语义结构,以推断接触发生的时间与位置。系统在多摄像机间重建全局一致的多人3D骨架轨迹,并从对齐深度数据的RGB图像中估计带有朝向的手部网格(MANO)。多视角手部重建结果融合后与追踪到的医护人员关联,获得一致的左右手轨迹。基于多视角分割与深度融合构建手术室的语义3D模型,使重建的手部轨迹可映射至具体表面,包括医疗设备、移动物体和患者体位。通过时间上的手-表面接近度推断接触事件,明确哪名医护人员在何时触碰了何种表面。我们在三个真实麻醉诱导过程的视频上评估了TouchMap-OR,手动标注了接触事件。结果表明,其接触二分类F1为0.75,优于基于跟踪的基线方法,同时保持相当的多人跟踪精度,并达到0.96的身份归属准确率。

原文摘要 · Abstract (English)

Hand-surface interactions between clinicians, patients, and medical equipment play a central role in pathogen transmission during medical procedures. However, these interactions remain largely unobserved, as current infection-prevention practices rely on manual observation and cannot reconstruct detailed contact histories. In this work we formulate the problem of identity-resolved hand-surface interaction reconstruction in operating rooms and introduce TouchMap-OR, a multi-view RGB-D vision system that models clinicians, articulated hand geometry, and the semantic structure of the clinical environment to infer when and where contacts occur. The system reconstructs globally consistent multi-person 3D skeleton tracks across cameras while estimating articulated MANO hand meshes from RGB observations aligned to depth data. Multi-view hand reconstructions are fused and associated with tracked clinicians to obtain consistent left and right hand trajectories. A semantic 3D model of the operating room is built from multi-view segmentation and depth fusion, enabling reconstructed hand trajectories to be mapped to specific surfaces, including medical equipment, movable objects, and patient body sites. Temporal hand-surface proximity is used to infer contact episodes describing which clinician touched which surface and when. We evaluate TouchMap-OR on recordings from three real anesthesia inductions with manually annotated contact events. TouchMap-OR achieves 0.75 binary contact F1, outperforming tracking-based baselines while maintaining comparable multi-person tracking accuracy and achieving 0.96 identity attribution accuracy.

3D重建手部追踪医疗安全多视角感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。