实时生成内镜视频中精准且时序一致的单目深度图
EndoStreamDepth: Temporally Consistent Monocular Depth Estimation for Endoscopic Video Streams
- 逐帧处理+时序模块，利用帧间信息提升稳定性
- 在两个结肠镜数据集上显著优于现有方法
- 适合机器人手术自动化等下游应用
本文提出EndoStreamDepth，一种针对内镜视频流的单目深度估计框架。该框架为每帧生成精确的深度图，保持清晰的解剖边界和帧间时序一致性，并实现实时推理。与以往使用批量输入的方法不同，EndoStreamDepth采用逐帧处理结合时序模块，传递帧间信息。其核心包含三个部分：(1) 基于内镜特化变换的单帧深度网络，生成高精度深度图；(2) 多层级Mamba时序模块，利用帧间信息提升准确性和预测稳定性；(3) 分层设计配合多尺度监督，通过互补损失项共同优化局部边界锐度与全局几何一致性。在两个公开可用的结肠镜深度估计数据集上进行综合评估，相比最先进的单目深度估计方法，EndoStreamDepth性能显著提升，生成的深度图具有清晰、解剖对齐的边界，对机器人手术自动化等下游任务至关重要。代码已公开于https://github.com/MedICL-VU/EndoStreamDepth。
原文摘要 · Abstract (English)
This work presents EndoStreamDepth, a monocular depth estimation framework for endoscopic video streams. It provides accurate depth maps with sharp anatomical boundaries for each frame, temporally consistent predictions across frames, and real-time throughput. Unlike prior work that uses batched inputs, EndoStreamDepth processes individual frames with a temporal module to propagate inter-frame information. The framework contains three main components: (1) a single-frame depth network with endoscopy-specific transformation to produce accurate depth maps, (2) multi-level Mamba temporal modules that leverage inter-frame information to improve accuracy and stabilize predictions, and (3) a hierarchical design with comprehensive multi-scale supervision, where complementary loss terms jointly improve local boundary sharpness and global geometric consistency. We conduct comprehensive evaluations on two publicly available colonoscopy depth estimation datasets. Compared to state-of-the-art monocular depth estimation methods, EndoStreamDepth substantially improves performance, and it produces depth maps with sharp, anatomically aligned boundaries, which are essential to support downstream tasks such as automation for robotic surgery. The code is publicly available at https://github.com/MedICL-VU/EndoStreamDepth
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。