实时流式视频深度估计，2K分辨率下每秒24帧
FlashDepth: Real-time Streaming Video Depth Estimation at 2K Resolution
- 基于预训练单图深度模型改进，轻量训练实现高精度
- 2044x1148分辨率下24帧/秒，边界更清晰
- 适合视频编辑、机器人实时决策等高要求场景
一个通用的视频深度估计模型应同时具备高精度、帧间一致性、高分辨率输出和实时流处理能力。本文提出FlashDepth，可在2044x1148分辨率的视频流上以24帧/秒的速度完成深度估计。通过针对预训练单图像深度模型进行精细调整，仅需少量数据和训练即可实现上述性能。我们在多个未见过的数据集上评估该方法，结果表明其在边界清晰度和速度方面显著优于现有先进模型，同时保持了具有竞争力的准确性。我们希望该模型能推动需要高分辨率深度信息的应用，如视频编辑与机器人在线决策。代码与模型权重已开源：https://github.com/Eyeline-Research/FlashDepth。
原文摘要 · Abstract (English)
A versatile video depth estimation model should (1) be accurate and consistent across frames, (2) produce high-resolution depth maps, and (3) support real-time streaming. We propose FlashDepth, a method that satisfies all three requirements, performing depth estimation on a 2044x1148 streaming video at 24 FPS. We show that, with careful modifications to pretrained single-image depth models, these capabilities are enabled with relatively little data and training. We evaluate our approach across multiple unseen datasets against state-of-the-art depth models, and find that ours outperforms them in terms of boundary sharpness and speed by a significant margin, while maintaining competitive accuracy. We hope our model will enable various applications that require high-resolution depth, such as video editing, and online decision-making, such as robotics. We release all code and model weights at https://github.com/Eyeline-Research/FlashDepth
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。