arXiv:2506.14856cs.CVcs.AI2025-06被引 4

用神经不确定性图自动选最佳视角,重建更准更快。

Peering into the Unknown: Active View Selection with Neural Uncertainty Maps for 3D Reconstruction

  • 用轻量网络直接从单图预测各视角的不确定性
  • 仅用一半视角达到同等重建精度,速度提升400倍
  • 无需额外训练即可泛化到新物体类别

某些视角天然比其他视角包含更多信息。如何让AI系统判断哪个视角最有助于精确高效地重建3D物体?主动视角选择(AVS)仍是计算机视觉中的基础挑战。目标是找到最少视角集合以获得最佳3D重建效果。我们提出一种新型AVS方法,不依赖于从当前观测中学习辐射场(如NeRF或3D Gaussian Splatting),而是通过一个轻量级前馈深度神经网络UPNet,基于单张输入图像输出预测的不确定性图,表示所有候选视角的不确定性值。该网络通过观察大量自然物体及其不确定性模式,学习从视角外观到潜在体素表示不确定性的直接映射。随后,我们的方法聚合先前预测的神经不确定性图,抑制冗余视角,有效选出最具信息量的视角。利用所选视角训练3D神经渲染模型,并与其它竞争性AVS方法对比新视角合成质量。令人惊讶的是,尽管使用的视角仅为上限的一半,本方法仍能达到相当的重建精度。此外,在AVS过程中显著降低计算开销,相比基线方法实现最高400倍的速度提升,同时减少50%以上的CPU、RAM和GPU使用。特别地,该方法可有效泛化至未见过的物体类别,且无需任何额外训练。

原文摘要 · Abstract (English)

Some perspectives naturally provide more information than others. How can an AI system determine which viewpoint offers the most valuable insight for accurate and efficient 3D object reconstruction? Active view selection (AVS) for 3D reconstruction remains a fundamental challenge in computer vision. The aim is to identify the minimal set of views that yields the most accurate 3D reconstruction. Instead of learning radiance fields, like NeRF or 3D Gaussian Splatting, from a current observation and computing uncertainty for each candidate viewpoint, we introduce a novel AVS approach guided by neural uncertainty maps predicted by a lightweight feedforward deep neural network, named UPNet. UPNet takes a single input image of a 3D object and outputs a predicted uncertainty map, representing uncertainty values across all possible candidate viewpoints. By leveraging heuristics derived from observing many natural objects and their associated uncertainty patterns, we train UPNet to learn a direct mapping from viewpoint appearance to uncertainty in the underlying volumetric representations. Next, our approach aggregates all previously predicted neural uncertainty maps to suppress redundant candidate viewpoints and effectively select the most informative one. Using these selected viewpoints, we train 3D neural rendering models and evaluate the quality of novel view synthesis against other competitive AVS methods. Remarkably, despite using half of the viewpoints than the upper bound, our method achieves comparable reconstruction accuracy. In addition, it significantly reduces computational overhead during AVS, achieving up to a 400 times speedup along with over 50\% reductions in CPU, RAM, and GPU usage compared to baseline methods. Notably, our approach generalizes effectively to AVS tasks involving novel object categories, without requiring any additional training.

3D重建主动选择不确定性估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。