不依赖额外信息,让视频单目深度估计更连贯。
STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimation
- 用表面法线方差区分静态与动态区域,分别处理
- 在KITTI和NYUv2上达到最新最好性能
- 适合需要稳定视频深度的自动驾驶等场景
视频单目深度估计对自动驾驶、AR/VR和机器人应用至关重要。基于Transformer的单图像深度模型在单帧上表现良好,但在视频帧间难以保持深度一致性。传统方法依赖多帧时序模块或光流、相机参数等先验信息,但存在内存高、动态运动下性能下降、运动理解有限等问题。本文提出STATIC,一种无需额外信息即可独立学习静态与动态区域时序一致性的新模型。通过表面法线的方向方差生成差异掩码,区分静态与动态区域:对静态区域使用掩码静态(MS)模块聚焦稳定区域;对动态区域使用表面法线相似性(SNS)模块,通过帧间特征相似性对齐并增强一致性。最终融合两部分独立学习结果,实现整段视频的时序一致性。在KITTI和NYUv2数据集上,STATIC无需额外信息即达到当前最优效果。
原文摘要 · Abstract (English)
Video monocular depth estimation is essential for applications such as autonomous driving, AR/VR, and robotics. Recent transformer-based single-image monocular depth estimation models perform well on single images but struggle with depth consistency across video frames. Traditional methods aim to improve temporal consistency using multi-frame temporal modules or prior information like optical flow and camera parameters. However, these approaches face issues such as high memory use, reduced performance with dynamic or irregular motion, and limited motion understanding. We propose STATIC, a novel model that independently learns temporal consistency in static and dynamic area without additional information. A difference mask from surface normals identifies static and dynamic area by measuring directional variance. For static area, the Masked Static (MS) module enhances temporal consistency by focusing on stable regions. For dynamic area, the Surface Normal Similarity (SNS) module aligns areas and enhances temporal consistency by measuring feature similarity between frames. A final refinement integrates the independently learned static and dynamic area, enabling STATIC to achieve temporal consistency across the entire sequence. Our method achieves state-of-the-art video depth estimation on the KITTI and NYUv2 datasets without additional information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。