用上下文条件让视频扩散模型精准估深度，仅需少量数据。
ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning
- 通过上下文条件控制，让预训练视频扩散模型直接做深度估计。
- 仅用80万帧训练，性能超越现有生成式方法，零样本泛化强。
- 适合追求高精度与低数据需求的视频深度估计研究者。
单目视频深度估计需兼顾时序一致性、几何精度和跨场景泛化能力，但现有方法难以同时实现。判别式模型虽单帧精度高，却因上下文窗口有限导致时序漂移；生成式方法虽提升一致性和泛化性，但需超1000万样本训练，且缺乏几何精度。为此，本文提出ICDepth框架，通过上下文条件（ICC）将预训练文本到视频扩散模型适配至视频深度估计任务，利用其丰富的时空先验。针对从生成任务迁移至密集预测的关键挑战，提出：(1) SAND-Attention，通过共享旋转位置编码实现精确时空对齐，并采用单向注意力防止噪声污染；(2) SRFM，注入DINOv2语义与分辨率先验以增强几何精度。ICDepth在多个基准上达到最先进性能，仅需0.8M帧训练（比同类生成方法少6–13倍），并展现出强大的零样本泛化能力。
原文摘要 · Abstract (English)
Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per-frame accuracy but suffer from temporal drift due to limited context windows, while generative methods improve consistency and generalization at the cost of extensive training data (10M+ samples) and lack of geometric precision. In response to these issues, we introduce \textbf{ICDepth}, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimation via In-Context Conditioning (ICC), leveraging their rich spatial-temporal priors. To address key challenges in transferring ICC from generation to dense prediction, we propose: (1)~\textbf{SAND-Attention}, which ensures precise spatial-temporal alignment via shared RoPE and enforces unidirectional attention to prevent noise contamination; (2)~\textbf{SRFM}, which injects DINOv2 semantic and resolution priors to enhance geometric precision. ICDepth achieves state-of-the-art results on multiple benchmarks with remarkable data efficiency, trained on only 0.8M frames ($6$--$13\times$ less than competing generative methods), while demonstrating strong zero-shot generalization to diverse domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。