arXiv:2605.18754cs.CV2026-05

检验3D生成模型在多视角下是否真实一致,发现现有方法易被噪声误导。

Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinate

论文配图:Can These Views Be One Scene? Evaluating Multiview 3D Consistency when 3D Foundation Models Hallucinate
图 1 · 摘自论文原文
  • 用几何验证对比神经重建先验,识别幻觉问题
  • 提出新基准和分解式指标,鲁棒性提升至3倍
  • 基于COLMAP的指标与人评相关性达MEt3R的4倍

多视角3D评估假设图像来自同一静态3D场景,但在神经视图合成(NVS)和稀疏视图重建中,输入或生成结果可能包含伪影、异常帧、重复视角或噪声,却仍获得高3D一致性评分。现有基于参考的度量需真值,而无真值度量如MEt3R依赖学习的重建骨干网络,其失效模式未被充分理解。本文通过比较神经重建先验与经典几何验证,研究该可靠性问题。引入enchmark,一个可控的鲁棒性评估基准,并提出参数化指标族,将神经度量分解为骨干、残差和聚合组件,可恢复MEt3R并生成鲁棒性最高达3倍的变体。分析表明,VGGT、MASt3R、DUSt3R和Fast3R可在无关场景、重复图像和随机噪声上幻觉出密集几何与跨视角支持。本文引入基于COLMAP的度量,利用匹配、注册、稠密支持及重建失败作为故障感知的一致性信号。在真实NVS输出和结构化人类研究中,这些度量与人工判断的相关性最高可达MEt3R的4倍。

原文摘要 · Abstract (English)

Multiview 3D evaluation assumes that the images being scored are observations of one static 3D scene. This assumption can fail in NVS and sparse-view reconstruction: inputs or generated outputs may contain artifacts, outlier frames, repeated views, or noise, yet still receive high 3D consistency scores. Existing reference-based metrics require ground truth, while ground-truth-free metrics such as MEt3R depend on learned reconstruction backbones whose failure modes are poorly characterized. We study this reliability problem by comparing neural reconstruction priors with classical geometric verification. We introduce \benchmark, a controlled robustness benchmark for multiview 3D consistency, and a parametric family that decomposes neural metrics into backbone, residual, and aggregation components. This family recovers MEt3R and yields variants up to $3\times$ more robust. Our analysis shows that VGGT, MASt3R, DUSt3R, and Fast3R can hallucinate dense geometry and cross-view support for unrelated scenes, repeated images, and random noise. We introduce COLMAP-based metrics that use matches, registration, dense support, and reconstruction failure as failure-aware consistency signals. On real NVS outputs and a structured human study, these metrics achieve up to $4\times$ higher correlation with human judgments than MEt3R.

3D生成多视角评估幻觉检测几何验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。