arXiv:2409.05442cs.CV2024-09被引 16

首个无需标注即可跨数据集实现内窥镜深度估计的模型,解决医疗数据少且质量差的问题。

EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labels

  • 用教师-学生框架自学习生成伪标签,结合噪声标签与置信度联合训练。
  • 在多个测试集上绝对相对误差比现有方法降低33%,零样本性能显著提升。
  • 适合医学影像、手术导航等需快速部署深度估计的场景,支持后续微调。

单图深度估计对内窥镜中的定位、重建和增强现实任务至关重要。现有方法多聚焦于域内估计,受限于医疗数据稀缺且标注质量差。本文提出EndoOmni,首个面向内窥镜的零样本跨域深度估计基础模型。为利用多样训练数据,采用先进自学习范式:教师模型生成伪标签,指导学生模型在大规模有标注与无标注数据上训练。针对深度标签固有噪声导致的训练干扰,提出鲁棒训练框架,联合使用深度标签与教师模型输出的置信度引导学生训练。进一步设计加权尺度-平移不变损失,根据标签置信度自适应调整学习权重,使模型更关注高质量像素,减弱噪声影响。零样本相对深度估计实验表明,EndoOmni在特定数据集上相较当前医学图像最佳方法降低33%绝对相对误差,较现有基础模型降低34%。同时提供良好初始化,支持微调以实现度量深度估计,在域内与域外场景均表现优异。代码已开源。

原文摘要 · Abstract (English)

Single-image depth estimation is essential for endoscopy tasks such as localization, reconstruction, and augmented reality. Most existing methods in surgical scenes focus on in-domain depth estimation, limiting their real-world applicability. This constraint stems from the scarcity and inferior labeling quality of medical data for training. In this work, we present EndoOmni, the first foundation model for zero-shot cross-domain depth estimation for endoscopy. To harness the potential of diverse training data, we refine the advanced self-learning paradigm that employs a teacher model to generate pseudo-labels, guiding a student model trained on large-scale labeled and unlabeled data. To address training disturbance caused by inherent noise in depth labels, we propose a robust training framework that leverages both depth labels and estimated confidence from the teacher model to jointly guide the student model training. Moreover, we propose a weighted scale-and-shift invariant loss to adaptively adjust learning weights based on label confidence, thus imposing learning bias towards cleaner label pixels while reducing the influence of highly noisy pixels. Experiments on zero-shot relative depth estimation show that our EndoOmni improves state-of-the-art methods in medical imaging for 33\% and existing foundation models for 34\% in terms of absolute relative error on specific datasets. Furthermore, our model provides strong initialization for fine-tuning metric depth estimation, maintaining superior performance in both in-domain and out-of-domain scenarios. The source code is publicly available at https://github.com/TianCuteQY/EndoOmni.

深度估计医学影像自学习零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。