将大模型缓存技术用于视频深度估计，实现低内存在线推理。
Online Video Depth Anything: Temporally-Consistent Depth Prediction with Low Memory Consumption
- 借鉴大模型缓存机制，动态保存特征避免重复计算。
- 在A100上达42帧/秒，Jetson设备上达20帧/秒，显存占用极低。
- 适合边缘设备部署，代码与编译脚本开源可直接使用。
单目视频的深度估计已成为众多实际计算机视觉系统的关键组件。近期，视频深度任意（Video Depth Anything, VDA）在长视频序列上表现出色，但其依赖批处理，无法用于在线场景。本文提出在线版VDA（oVDA），核心创新在于借鉴大语言模型中的技术：推理时缓存潜在特征，训练时对帧进行掩码。oVDA在准确率和显存占用上均优于现有在线视频深度估计方法。尤其在边缘设备部署中表现优异，可在NVIDIA A100上实现42 FPS，在NVIDIA Jetson设备上实现20 FPS。我们还将开源代码与编译脚本，便于在低功耗硬件上快速部署。
原文摘要 · Abstract (English)
Depth estimation from monocular video has become a key component of many real-world computer vision systems. Recently, Video Depth Anything (VDA) has demonstrated strong performance on long video sequences. However, it relies on batch-processing which prohibits its use in an online setting. In this work, we overcome this limitation and introduce online VDA (oVDA). The key innovation is to employ techniques from Large Language Models (LLMs), namely, caching latent features during inference and masking frames at training. Our oVDA method outperforms all competing online video depth estimation methods in both accuracy and VRAM usage. Low VRAM usage is particularly important for deployment on edge devices. We demonstrate that oVDA runs at 42 FPS on an NVIDIA A100 and at 20 FPS on an NVIDIA Jetson edge device. We will release both, code and compilation scripts, making oVDA easy to deploy on low-power hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。