通过融合探头位置与扫查内相似性,提升术中超声图像检索准确率。
Image Retrieval with Intra-Sweep Representation Learning for Neck Ultrasound Scanning Guidance
- 利用探头位置和扫查内相似性设计自监督对比学习策略
- 模拟数据上检索准确率达92.30%,优于现有时序对比方法
- 适用于术中实时引导,尤其适合舌部牵拉后组织变形场景
目的:术中超声(US)可增强经口机器人手术中的实时可视化。外科医生基于术前扫描建立心理地图,术中由助手自由手持超声扫查,医生在远程操作台进行手术。如何将目标扫查平面传达给医生存在困难。自动图像检索可将术中图像与术前图像库匹配,指导助手调整探头方向。方法:我们提出一种自监督对比学习方法,用于匹配术中超声视图与术前图像数据库。引入一种新颖的对比学习策略,利用扫查内相似性和超声探头位置来优化特征编码。同时,模型采用灵活阈值拒绝不理想匹配。结果:在模拟数据上,本方法检索准确率达92.30%,优于现有基于时间的对比学习方法。消融实验表明,在优化目标中引入探头位置可提升图像表征能力,说明探头位置可提供语义信息。我们在真实患者数据上验证了该超声探头定位系统的可行性,尽管存在舌部牵拉导致的组织形变。结论:结合扫查内相似性和探头位置的对比学习方法,增强了超声图像表征学习能力。我们还展示了该图像检索方法在舌部牵拉后的实际超声数据上实现颈部定位的可行性。
原文摘要 · Abstract (English)
Purpose: Intraoperative ultrasound (US) can enhance real-time visualization in transoral robotic surgery. The surgeon creates a mental map with a pre-operative scan. Then, a surgical assistant performs freehand US scanning during the surgery while the surgeon operates at the remote surgical console. Communicating the target scanning plane in the surgeon's mental map is difficult. Automatic image retrieval can help match intraoperative images to preoperative scans, guiding the assistant to adjust the US probe toward the target plane. Methods: We propose a self-supervised contrastive learning approach to match intraoperative US views to a preoperative image database. We introduce a novel contrastive learning strategy that leverages intra-sweep similarity and US probe location to improve feature encoding. Additionally, our model incorporates a flexible threshold to reject unsatisfactory matches. Results: Our method achieves 92.30% retrieval accuracy on simulated data and outperforms state-of-the-art temporal-based contrastive learning approaches. Our ablation study demonstrates that using probe location in the optimization goal improves image representation, suggesting that semantic information can be extracted from probe location. We also present our approach on real patient data to show the feasibility of the proposed US probe localization system despite tissue deformation from tongue retraction. Conclusion: Our contrastive learning method, which utilizes intra-sweep similarity and US probe location, enhances US image representation learning. We also demonstrate the feasibility of using our image retrieval method to provide neck US localization on real patient US after tongue retraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。