让视觉语言模型直接生成高精度稠密深度图，一次推理完成图文输出。
Unlocking Dense Metric Depth Estimation in VLMs
- 在大语言模型主干上加轻量深度头，统一图文监督训练。
- 单次前向传播生成全分辨率深度图，效率高于现有方法。
- 提升复杂3D空间推理能力，适合多模态基础模型研究者。
视觉语言模型（VLM）在2D任务如图像定位和描述方面表现优异，但在3D理解上仍受限。主要瓶颈在于仅依赖文本监督，导致细粒度视觉感知不足，难以恢复稠密几何结构。此前方法要么从外部视觉模型蒸馏几何信息（引入误差累积），要么通过低效的逐像素查询或粗粒度标记输出实现直接预测。本文提出DepthVLM，一种简单有效的框架，将单一VLM转变为原生稠密几何预测器，同时保持其多模态能力。通过在LLM主干上附加轻量级深度头，并采用两阶段联合视觉-文本监督训练策略，DepthVLM可在一次前向传播中生成完整分辨率深度图与语言输出。我们进一步构建了一个兼容VLM格式的统一室内外度量深度基准测试集。实验表明，DepthVLM显著优于现有VLM，推理效率更高，超越主流纯视觉模型，在复杂3D空间推理任务中表现更优，推动迈向真正统一的多模态基础模型。项目主页：https://depthvlm.github.io/
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) excel at 2D tasks such as grounding and captioning, yet remain limited in 3D understanding. A key limitation is their text-only supervision paradigm, which under-constrains fine-grained visual perception and prevents the recovery of dense geometry. Prior methods either distill geometry from external vision models, introducing error accumulation, or enable direct prediction with inefficient per-pixel query or coarse token-level outputs. In this paper, we propose DepthVLM, a simple yet effective framework that transforms a single VLM into a native dense geometry predictor while preserving its multimodal capability. By attaching a lightweight depth head to the LLM backbone and training under a unified vision-text supervision paradigm with a two-stage schedule, DepthVLM generates full-resolution depth maps alongside language outputs in a single forward pass. We further introduce a unified indoor-outdoor metric depth benchmark in a VLM-compatible format. Experiments show that DepthVLM significantly outperforms existing VLMs with higher inference efficiency, surpasses leading pure vision models, and improves complex 3D spatial reasoning, moving toward a truly unified multimodal foundation model. The project page is available at https://depthvlm.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。