用冻结的CLIP模型实现轻量级单目深度估计，无需文本提示。
SPACE-CLIP: Spatial Perception via Adaptive CLIP Embeddings for Monocular Depth Estimation
- 仅用冻结的CLIP视觉编码器，直接解析几何特征。
- KITTI上AbsRel达0.0901，NYU Depth V2上为0.1042。
- 可适配多种基础模型，适合嵌入式机器人系统部署。
机器人与自主系统需要密集的空间线索，但许多单目深度模型体积庞大、任务特定或难以集成到现有多模态架构中。CLIP 提供强大的语义表示，然而大多数基于 CLIP 的深度方法仍依赖文本提示或主干网络更新，这增加了在集成控制流水线中部署的复杂性。我们提出 SPACE-CLIP，一种仅使用解码器的深度框架，直接从冻结的 CLIP 视觉编码器读取几何线索，并在推理时跳过文本编码器。该模型将深层的 FiLM 条件语义特征与浅层的结构特征结合，以恢复全局场景布局和局部几何细节。在无文本提示（text-free inference）且视觉主干冻结（frozen vision backbone）的 TFI-FB 约束下，SPACE-CLIP 在 KITTI 上达到 AbsRel 0.0901，在 NYU Depth V2 上为 0.1042；相同双路径解码器也可迁移至冻结的 SigLIP 主干并获得相当性能。结果表明，一个紧凑的解码器可将共享的基础模型主干转化为可复用的空间感知模块，适用于具身智能与自主机器人系统。代码已开源：https://github.com/taewan2002/space-clip
原文摘要 · Abstract (English)
Robotic and autonomous systems need dense spatial cues, but many monocular depth models are heavy, task-specific, or hard to attach to an existing multimodal stack. CLIP offers strong semantic representations, yet most CLIP-based depth methods still depend on text prompts or backbone updates, which complicate deployment in integrated control pipelines. We present SPACE-CLIP, a decoder-only depth framework that reads geometric cues directly from a frozen CLIP vision encoder and bypasses the text encoder at inference time. The model combines FiLM-conditioned semantic features from deep layers with structural features from shallow layers to recover both global scene layout and local geometric detail. Under the TFI-FB constraint (text-free inference and frozen vision backbone), SPACE-CLIP achieves AbsRel 0.0901 on KITTI and 0.1042 on NYU Depth V2, and the same dual-pathway decoder transfers to a frozen SigLIP backbone with comparable results. These findings show that a compact decoder can turn a shared foundation-model backbone into a reusable spatial perception module for embodied AI and autonomous robotic systems. Our model is available at https://github.com/taewan2002/space-clip
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。