用海量杂乱数据训练出能通用的度量深度模型,打破传感器和相机限制。
MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources
- 通过随机遮蔽深度图生成通用提示,分离空间推理与设备偏差。
- 在2000万图像-深度对上实现度量深度的清晰缩放规律。
- 适用于单目深度、3D重建、多模态大模型空间理解等任务。
缩放推动了视觉基础模型的进展,但将此范式扩展到度量深度估计仍面临异构传感器噪声、相机依赖偏差及跨源3D数据的度量模糊性挑战。我们提出Metric Anything,一种简单可扩展的预训练框架,无需人工设计提示、相机特异性建模或任务专用架构,即可从噪声多样化的3D数据中学习度量深度。核心是稀疏度量提示(Sparse Metric Prompt),通过随机遮蔽深度图生成,作为解耦空间推理与传感器/相机偏差的通用接口。利用约2000万张图像-深度对,涵盖10000种相机型号的重建、采集和渲染3D数据,首次展示了度量深度任务中的明确缩放趋势。预训练模型在提示驱动任务(如深度补全、超分辨率、雷达-相机融合)中表现优异;其轻量化无提示学生模型在单目深度估计、相机内参恢复、单/多视图度量3D重建及视觉-语言-动作(VLA)规划任务中达到当前最优。此外,将Metric Anything的ViT作为视觉编码器可显著提升多模态大模型的空间智能能力。结果表明,度量深度估计可受益于驱动现代基础模型的相同缩放定律,为可扩展、高效的现实世界度量感知开辟新路径。代码已开源:http://metric-anything.github.io/metric-anything-io/
原文摘要 · Abstract (English)
Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data. We introduce Metric Anything, a simple and scalable pretraining framework that learns metric depth from noisy, diverse 3D sources without manually engineered prompts, camera-specific modeling, or task-specific architectures. Central to our approach is the Sparse Metric Prompt, created by randomly masking depth maps, which serves as a universal interface that decouples spatial reasoning from sensor and camera biases. Using about 20M image-depth pairs spanning reconstructed, captured, and rendered 3D data across 10000 camera models, we demonstrate-for the first time-a clear scaling trend in the metric depth track. The pretrained model excels at prompt-driven tasks such as depth completion, super-resolution and Radar-camera fusion, while its distilled prompt-free student achieves state-of-the-art results on monocular depth estimation, camera intrinsics recovery, single/multi-view metric 3D reconstruction, and VLA planning. We also show that using pretrained ViT of Metric Anything as a visual encoder significantly boosts Multimodal Large Language Model capabilities in spatial intelligence. These results show that metric depth estimation can benefit from the same scaling laws that drive modern foundation models, establishing a new path toward scalable and efficient real-world metric perception. We open-source MetricAnything at http://metric-anything.github.io/metric-anything-io/ to support community research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。