纯基于CLIP空间的单目深度估计，无需提示词和解码器
PureCLIP-Depth: Prompt-Free and Decoder-Free Monocular Depth Estimation within CLIP Embedding Space
- 在CLIP嵌入空间内直接映射图像到深度，不依赖几何特征
- 在室内外数据集上均达到当前基于CLIP模型的最优性能
- 适合追求轻量级、无需额外训练提示或解码结构的场景
我们提出PureCLIP-Depth，一种完全无需提示词、无需解码器的单目深度估计（MDE）模型，其所有计算均在对比语言-图像预训练（CLIP）嵌入空间内完成。与依赖几何特征的现有方法不同，本方法通过概念信息驱动深度推断，在概念空间中直接完成从RGB到深度的映射。该方法在室内和室外数据集上均实现了基于CLIP嵌入模型的最先进性能。相关代码已公开于https://github.com/ryutaroLF/PureCLIP-Depth。
原文摘要 · Abstract (English)
We propose PureCLIP-Depth, a completely prompt-free, decoder-free Monocular Depth Estimation (MDE) model that operates entirely within the Contrastive Language-Image Pre-training (CLIP) embedding space. Unlike recent models that rely heavily on geometric features, we explore a novel approach to MDE driven by conceptual information, performing computations directly within the conceptual CLIP space. The core of our method lies in learning a direct mapping from the RGB domain to the depth domain strictly inside this embedding space. Our approach achieves state-of-the-art performance among CLIP embedding-based models on both indoor and outdoor datasets. The code used in this research is available at: https://github.com/ryutaroLF/PureCLIP-Depth
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。