用轻量提示引导适配，让CLIP模型更准地估计单目深度。
Lightweight Prompt-Guided CLIP Adaptation for Monocular Depth Estimation
- 引入轻量混合适配器模块，结合语义提示实现空间感知适配。
- 在NYU Depth V2上将误差降低至0.520，精度提升至0.745。
- 参数极少却效果显著，适合资源受限的深度估计应用。
利用视觉语言模型（如CLIP）丰富的语义特征进行单目深度估计具有潜力，但通常需要大量微调或几何精度不足。本文提出一种参数高效框架MoA-DepthCLIP，通过在预训练ViT-B/32主干网络中集成轻量级混合适配器（MoA）模块，并对最后几层进行选择性微调，实现基于全局语义上下文向量的时空感知适配。该方法采用深度分类与直接回归结合的混合预测架构，配合施加几何约束的复合损失函数，提升结构准确性。在NYU Depth V2基准测试中，相比DepthCLIP基线，$δ_1$精度从0.390提升至0.745，均方根误差（RMSE）由1.176降至0.520，同时仅需极少可训练参数，证明了轻量、提示引导的MoA是迁移视觉语言模型知识到细粒度单目深度估计任务的有效策略。
原文摘要 · Abstract (English)
Leveraging the rich semantic features of vision-language models (VLMs) like CLIP for monocular depth estimation tasks is a promising direction, yet often requires extensive fine-tuning or lacks geometric precision. We present a parameter-efficient framework, named MoA-DepthCLIP, that adapts pretrained CLIP representations for monocular depth estimation with minimal supervision. Our method integrates a lightweight Mixture-of-Adapters (MoA) module into the pretrained Vision Transformer (ViT-B/32) backbone combined with selective fine-tuning of the final layers. This design enables spatially-aware adaptation, guided by a global semantic context vector and a hybrid prediction architecture that synergizes depth bin classification with direct regression. To enhance structural accuracy, we employ a composite loss function that enforces geometric constraints. On the NYU Depth V2 benchmark, MoA-DepthCLIP achieves competitive results, significantly outperforming the DepthCLIP baseline by improving the $δ_1$ accuracy from 0.390 to 0.745 and reducing the RMSE from 1.176 to 0.520. These results are achieved while requiring substantially few trainable parameters, demonstrating that lightweight, prompt-guided MoA is a highly effective strategy for transferring VLM knowledge to fine-grained monocular depth estimation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。