一次推理完成3D重建与语义理解，支持任意视角输入。
Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images
- 用跨视图变换器融合多视角图像，生成带语义的3D高斯点。
- 在RE10K上达到25.07 PSNR，ScanNet上达55.84 mIoU。
- 无需逐场景优化，适合大规模通用3D理解任务。
从稀疏2D视图中统一重建并语义解析3D场景仍是计算机视觉的核心挑战。传统方法常将语义理解与重建分离，或需昂贵的逐场景优化，限制了可扩展性与泛化能力。本文提出Uni3R，一种前馈式框架，直接从无姿态多视角图像中联合重建包含开放词汇语义的统一3D场景表示。该方法利用跨视图变换器鲁棒融合任意多视角输入，回归一组带有语义特征场的3D高斯基元。此统一表示支持高保真新视角合成、开放词汇3D语义分割与深度预测，均在单次前馈中完成。大量实验表明，Uni3R在多个基准上达到新最佳性能，包括RE10K上的25.07 PSNR和ScanNet上的55.84 mIoU。本工作标志着通用化、统一化3D场景重建与理解的新范式。代码已公开于https://github.com/HorizonRobotics/Uni3R。
原文摘要 · Abstract (English)
Reconstructing and semantically interpreting 3D scenes from sparse 2D views remains a fundamental challenge in computer vision. Conventional methods often decouple semantic understanding from reconstruction or necessitate costly per-scene optimization, thereby restricting their scalability and generalizability. In this paper, we introduce Uni3R, a novel feed-forward framework that jointly reconstructs a unified 3D scene representation enriched with open-vocabulary semantics, directly from unposed multi-view images. Our approach leverages a Cross-View Transformer to robustly integrate information across arbitrary multi-view inputs, which then regresses a set of 3D Gaussian primitives endowed with semantic feature fields. This unified representation facilitates high-fidelity novel view synthesis, open-vocabulary 3D semantic segmentation, and depth prediction, all within a single, feed-forward pass. Extensive experiments demonstrate that Uni3R establishes a new state-of-the-art across multiple benchmarks, including 25.07 PSNR on RE10K and 55.84 mIoU on ScanNet. Our work signifies a novel paradigm towards generalizable, unified 3D scene reconstruction and understanding. The code is available at https://github.com/HorizonRobotics/Uni3R.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。