从单图生成可自由探索的3D场景,解决几何失真问题
One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Image
- 分三步走:先生成全景图,再建3D几何骨架,最后生成任意视角图像
- 在大视角移动下仍保持几何一致,深度估计误差比现有方法低27%
- 适合需要高保真3D重建的虚拟现实、自动驾驶等场景
从单张图像生成可自由探索的3D场景是3D视觉中的难题。现有方法在远距离视角变化时易产生严重几何畸变和噪声。本文提出One2Scene,将该病态问题分解为三个可处理子任务。首先用全景生成器从输入图像生成初始锚点视图;接着通过通用前馈高斯点云网络,将这些2D锚点提升为显式3D几何骨架;不同于将全景视为单图重建,我们将其投影为多个稀疏锚点视图,并重构为多视图立体匹配任务,从而利用大规模多视角数据学习的鲁棒几何先验。采用双向特征融合模块增强跨视图一致性,获得高效且几何可靠的骨架。最后,以该骨架为强先验,使用新型视图生成器在任意相机位置生成逼真且几何准确的视图。通过显式依赖3D一致骨架进行重建,One2Scene在大视角运动下依然稳定,支持沉浸式场景探索。大量实验表明,One2Scene在全景深度估计、前馈360°重建和可探索3D场景生成上均显著优于当前最佳方法。
原文摘要 · Abstract (English)
Generating explorable 3D scenes from a single image is a highly challenging problem in 3D vision. Existing methods struggle to support free exploration, often producing severe geometric distortions and noisy artifacts when the viewpoint moves far from the original perspective. We introduce \textbf{One2Scene}, an effective framework that decomposes this ill-posed problem into three tractable sub-tasks to enable immersive explorable scene generation. We first use a panorama generator to produce anchor views from a single input image as initialization. Then, we lift these 2D anchors into an explicit 3D geometric scaffold via a generalizable, feed-forward Gaussian Splatting network. Instead of treating the panorama as a single image for reconstruction, we project it into multiple sparse anchor views and reformulate the reconstruction task as multi-view stereo matching, which allows us to leverage robust geometric priors learned from large-scale multi-view datasets. A bidirectional feature fusion module is used to enforce cross-view consistency, yielding an efficient and geometrically reliable scaffold. Finally, the scaffold serves as a strong prior for a novel view generator to produce photorealistic and geometrically accurate views at arbitrary cameras. By explicitly conditioning on a 3D-consistent scaffold to perform reconstruction, One2Scene works stably under large camera motions, supporting immersive scene exploration. Extensive experiments show that One2Scene substantially outperforms state-of-the-art methods in panorama depth estimation, feed-forward 360° reconstruction, and explorable 3D scene generation. Project page: https://one2scene5406.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。