HoloLens2上实现零样本自动标注，速度提升500倍
HOLa: HoloLens Object Labeling
- 基于SAM-Track算法，全自动追踪单个目标并生成分割掩码
- 标注速度提升500倍以上，Dice分数达0.875~0.982
- 无需调整即可适配不同场景，适合医学AR研究者
在医疗增强现实（AR）应用中，目标跟踪面临标注掩码需求量大的挑战。随着分割基础模型（如分割任意模型SAM）的出现，零样本分割仅需极少人工参与即可获得高质量对象掩码。我们提出基于SAM-Track算法的HoloLens-Object-Labeling（HOLa）Unity与Python应用，可在HoloLens 2上实现单对象标注的全自动化，且人类干预极少。HOLa无需针对特定图像外观进行调整，可广泛适用于各类AR研究场景。我们在开放肝手术和医学模拟实验中评估了HOLa在不同图像复杂度下的表现。使用HOLa进行图像标注可使标注速度提升超过500倍，同时获得0.875至0.982之间的Dice分数，与人工标注相当。代码已公开：https://github.com/mschwimmbeck/HOLa
原文摘要 · Abstract (English)
In the context of medical Augmented Reality (AR) applications, object tracking is a key challenge and requires a significant amount of annotation masks. As segmentation foundation models like the Segment Anything Model (SAM) begin to emerge, zero-shot segmentation requires only minimal human participation obtaining high-quality object masks. We introduce a HoloLens-Object-Labeling (HOLa) Unity and Python application based on the SAM-Track algorithm that offers fully automatic single object annotation for HoloLens 2 while requiring minimal human participation. HOLa does not have to be adjusted to a specific image appearance and could thus alleviate AR research in any application field. We evaluate HOLa for different degrees of image complexity in open liver surgery and in medical phantom experiments. Using HOLa for image annotation can increase the labeling speed by more than 500 times while providing Dice scores between 0.875 and 0.982, which are comparable to human annotators. Our code is publicly available at: https://github.com/mschwimmbeck/HOLa
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。