arXiv:2506.23721eess.IVcs.AI2025-06

用深度学习与增强现实,实现超声实时自动测肾体积

Deep Learning-Based Semantic Segmentation for Real-Time Kidney Imaging and Measurements with Augmented Reality-Assisted Ultrasound

  • 结合深度学习与增强现实,实现超声图像的实时语义分割
  • 在Open Kidney数据集上达到高精度肾体积测量,延迟低于300ms
  • 支持多种设备,适合床旁诊疗和超声教学场景

超声检查虽无辐射且易获取,但因动态成像和非标准切面导致学习曲线陡峭,且医生需频繁在屏幕与患者间切换注意力。为解决此问题,本文提出基于深度学习的实时(RT)语义分割方法,实现肾脏体积自动化测量,该过程传统上耗时且易疲劳。该自动化使医生可专注图像解读而非手动测量。同时,增强现实(AR)将显示投射至医生视野中,提升操作舒适度并减少认知负荷。本文在HoloLens-2上设计两种AR-DL超声系统:一种通过API无线流式传输,另一种兼容任意带视频输出的超声设备,拓展适用范围。基于Open Kidney数据集及nnU-Net、Segmenter、YOLO with MedSAM和LiteMedSAM等开源模型评估实时性与准确性。开源项目包含模型实现、测量算法与基于Wi-Fi的流媒体方案,助力超声培训与床旁诊断。

原文摘要 · Abstract (English)

Ultrasound (US) is widely accessible and radiation-free but has a steep learning curve due to its dynamic nature and non-standard imaging planes. Additionally, the constant need to shift focus between the US screen and the patient poses a challenge. To address these issues, we integrate deep learning (DL)-based semantic segmentation for real-time (RT) automated kidney volumetric measurements, which are essential for clinical assessment but are traditionally time-consuming and prone to fatigue. This automation allows clinicians to concentrate on image interpretation rather than manual measurements. Complementing DL, augmented reality (AR) enhances the usability of US by projecting the display directly into the clinician's field of view, improving ergonomics and reducing the cognitive load associated with screen-to-patient transitions. Two AR-DL-assisted US pipelines on HoloLens-2 are proposed: one streams directly via the application programming interface for a wireless setup, while the other supports any US device with video output for broader accessibility. We evaluate RT feasibility and accuracy using the Open Kidney Dataset and open-source segmentation models (nnU-Net, Segmenter, YOLO with MedSAM and LiteMedSAM). Our open-source GitHub pipeline includes model implementations, measurement algorithms, and a Wi-Fi-based streaming solution, enhancing US training and diagnostics, especially in point-of-care settings.

超声语义分割增强现实实时测量

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。