arXiv:2506.01933cs.CV2025-06被引 8

首个端到端3D几何基础模型评测基准,助力三维智能研究

E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Models

  • 构建涵盖5项核心任务的统一评测框架
  • 评估16个先进模型在标准与挑战数据集上的表现
  • 开源工具链支持公平可复现的对比研究

空间智能涵盖3D重建、感知与推理,是机器人、航拍和扩展现实等应用的基础。关键在于从非结构化或流式图像中实时准确估计核心3D属性(相机参数、点云、深度图、3D点轨迹)。受大语言模型和2D视觉基础模型成功的启发,端到端3D几何基础模型(GFMs)应运而生,可在单次前向传播中直接预测稠密3D表示,无需依赖缓慢或缺失的预计算相机参数。自2023年底以来,该领域迅速发展出多种变体,但缺乏系统性评估。本文提出首个针对3D GFMs的综合性评测基准,覆盖五项核心任务:稀疏视图深度估计、视频深度估计、3D重建、多视角位姿估计、新视角合成,并涵盖标准与分布外挑战数据集。标准化工具链自动化处理数据集、评估协议与指标计算,确保公平、可复现的比较。我们评估了16个前沿GFMs,揭示其在不同任务与域中的优劣,并提炼出指导未来模型扩展与优化的关键洞察。所有代码、评估脚本与处理后的数据将公开发布,以加速3D空间智能研究。

原文摘要 · Abstract (English)

Spatial intelligence, encompassing 3D reconstruction, perception, and reasoning, is fundamental to applications such as robotics, aerial imaging, and extended reality. A key enabler is the real-time, accurate estimation of core 3D attributes (camera parameters, point clouds, depth maps, and 3D point tracks) from unstructured or streaming imagery. Inspired by the success of large foundation models in language and 2D vision, a new class of end-to-end 3D geometric foundation models (GFMs) has emerged, directly predicting dense 3D representations in a single feed-forward pass, eliminating the need for slow or unavailable precomputed camera parameters. Since late 2023, the field has exploded with diverse variants, but systematic evaluation is lacking. In this work, we present the first comprehensive benchmark for 3D GFMs, covering five core tasks: sparse-view depth estimation, video depth estimation, 3D reconstruction, multi-view pose estimation, novel view synthesis, and spanning both standard and challenging out-of-distribution datasets. Our standardized toolkit automates dataset handling, evaluation protocols, and metric computation to ensure fair, reproducible comparisons. We evaluate 16 state-of-the-art GFMs, revealing their strengths and limitations across tasks and domains, and derive key insights to guide future model scaling and optimization. All code, evaluation scripts, and processed data will be publicly released to accelerate research in 3D spatial intelligence.

3D几何基础模型评测基准空间智能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。