用时空特征传播提升视频深度估计的实时性与一致性
Video Depth Propagation
- 通过光流变形+残差修正实现深度特征跨帧传播
- 在多个基准上达到最优时序一致性，推理速度显著更快
- 适合需要实时深度感知的视觉任务应用
视频深度估计对真实场景中的视觉感知至关重要。现有方法或采用简单逐帧单目模型，导致时间不一致和误差；或依赖计算量大的时序建模，难以满足实时需求。为此，我们提出 VeloDepth，一种高效且鲁棒的在线视频深度估计框架，有效利用历史深度预测的时空先验并进行深度特征传播。该方法引入新颖的传播模块，通过基于光流的变形与学习到的残差修正来精炼和传播深度特征与预测。设计上结构化地保证了时序一致性，使连续帧间深度预测更稳定且效率更高。在多个基准上的零样本评估表明，VeloDepth 在时序一致性方面达到当前最佳水平，同时保持竞争性精度，并显著快于现有视频深度估计算法。VeloDepth 为多样化感知任务提供了实用、高效且准确的实时深度估计方案。代码与模型已开源。
原文摘要 · Abstract (English)
Depth estimation in videos is essential for visual perception in real-world applications. However, existing methods either rely on simple frame-by-frame monocular models, leading to temporal inconsistencies and inaccuracies, or use computationally demanding temporal modeling, unsuitable for real-time applications. These limitations significantly restrict general applicability and performance in practical settings. To address this, we propose VeloDepth, an efficient and robust online video depth estimation pipeline that effectively leverages spatiotemporal priors from previous depth predictions and performs deep feature propagation. Our method introduces a novel Propagation Module that refines and propagates depth features and predictions using flow-based warping coupled with learned residual corrections. In addition, our design structurally enforces temporal consistency, resulting in stable depth predictions across consecutive frames with improved efficiency. Comprehensive zero-shot evaluation on multiple benchmarks demonstrates the state-of-the-art temporal consistency and competitive accuracy of VeloDepth, alongside its significantly faster inference compared to existing video-based depth estimators. VeloDepth thus provides a practical, efficient, and accurate solution for real-time depth estimation suitable for diverse perception tasks. Code and models are available at https://github.com/lpiccinelli-eth/velodepth
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。