arXiv:2605.10525cs.CV2026-05被引 1

通过嵌入相机运动信息,提升视频深度估计的3D一致性与细节清晰度。

GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth

论文配图:GemDepth: Geometry-Embedded Features for 3D-Consistent Video Depth
图 1 · 摘自论文原文
  • 引入几何嵌入模块,利用相机位姿预测生成隐式几何特征。
  • 在多个数据集上达到当前最优性能,尤其在动态复杂场景中表现突出。
  • 适合需要高精度视频深度估计的研究者和工业应用开发者。

视频深度估计将单目预测扩展到时序域以保证一致性,但现有方法常在细节区域出现空间模糊和时间不一致问题。我们认为,依赖Transformer进行时序平滑的方法难以维持严格的3D几何一致性,尤其是在旋转或剧烈视角变化下。为此,我们提出GemDepth,其核心思想是显式感知相机运动与全局3D结构是实现3D一致性的前提。不同于以往方法,GemDepth引入几何嵌入模块(GEM),通过预测帧间相机位姿生成隐式几何嵌入,赋予网络内在的3D感知与对齐能力。在此基础上,交替时空变换器(ASTT)捕捉潜在点级对应关系,同时提升空间精度以保留细节,并强化时间一致性。此外,该框架采用高效训练策略,有效平衡了效率与几何鲁棒性。如图2所示,全面评估表明,GemDepth在多个数据集上均取得领先性能,尤其在复杂动态场景中优势显著。代码已公开于:https://github.com/Yuecheng919/GemDepth。

原文摘要 · Abstract (English)

Video depth estimation extends monocular prediction into the temporal domain to ensure coherence. However, existing methods often suffer from spatial blurring in fine-detail regions and temporal inconsistencies. We argue that current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency-particularly under rotations or drastic view changes. To address this, we propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency. Distinctively, GemDepth introduces a Geometry-Embedding Module (GEM) that predicts inter-frame camera poses to generate implicit geometric embeddings. This injection of motion priors equips the network with intrinsic 3D perception and alignment capabilities. Guided by these geometric cues, our Alternating Spatio-Temporal Transformer (ASTT) captures latent point-level correspondences to simultaneously enhance spatial precision for sharp details and enforce rigorous temporal consistency. Furthermore, GemDepth employs a data-efficient training strategy, effectively bridging the gap between high efficiency and robust geometric consistency. As shown in Fig.2, comprehensive evaluations demonstrate that GemDepth achieves state-of-the-art performance across multiple datasets, particularly in complex dynamic scenarios. The code is publicly available at: https://github.com/Yuecheng919/GemDepth.

视频深度3D一致性几何先验Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。