用生成模型先验实现视频深度确定性估计,不依赖大量标注数据。
DVD: Deterministic Video Depth Estimation with Generative Priors
- 将扩散模型时间步重用于结构锚定,平衡全局稳定与细节精度。
- 通过隐空间校正缓解回归过平滑,恢复清晰边界与连贯运动。
- 利用全局仿射一致性实现长视频无缝推断,仅需163倍少数据。
现有视频深度估计面临根本矛盾:生成模型易产生随机几何幻觉和尺度漂移,判别模型则需海量标注数据解决语义模糊。为此,我们提出DVD,首个将预训练视频扩散模型确定性转为单次前向深度回归器的框架。核心设计包括:(i) 将扩散时间步作为结构锚点,兼顾全局稳定性与高频细节;(ii) 隐空间流形校正(LMR),通过微分约束缓解回归导致的过平滑,恢复锐利边界与一致运动;(iii) 全局仿射一致性,天然抑制帧间偏差,支持无需复杂时序对齐的长视频无缝推理。大量实验表明,DVD在多个基准上实现零样本最佳性能,且仅需领先基线163倍少的任务特定数据,即可激活视频基础模型中的深层几何先验。我们已完整开源整个训练流程,助力开源社区达成当前最优视频深度估计。
原文摘要 · Abstract (English)
Existing video depth estimation faces a fundamental trade-off: generative models suffer from stochastic geometric hallucinations and scale drift, while discriminative models demand massive labeled datasets to resolve semantic ambiguities. To break this impasse, we present DVD, the first framework to deterministically adapt pre-trained video diffusion models into single-pass depth regressors. Specifically, DVD features three core designs: (i) repurposing the diffusion timestep as a structural anchor to balance global stability with high-frequency details; (ii) latent manifold rectification (LMR) to mitigate regression-induced over-smoothing, enforcing differential constraints to restore sharp boundaries and coherent motion; and (iii) global affine coherence, an inherent property bounding inter-window divergence, which enables seamless long-video inference without requiring complex temporal alignment. Extensive experiments demonstrate that DVD achieves state-of-the-art zero-shot performance across benchmarks. Furthermore, DVD successfully unlocks the profound geometric priors implicit in video foundation models using 163x less task-specific data than leading baselines. Notably, we fully release our pipeline, providing the whole training suite for SOTA video depth estimation to benefit the open-source community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。