仅用一张全景图生成可自由漫游的高保真3D室内场景。
Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation
- 通过可控轨迹的全景视频建模，实现无需多视角的3D生成。
- 4步去噪生成高质量视频，重建3D场景支持实时自由视角漫游。
- 零样本泛化到未见室内场景，适合智能体仿真应用。
我们解决仅从单张360°全景图重建高保真、可自由漫游3D场景的问题，无需每场景优化或多视角采集。现有方法或缺乏度量级轨迹控制，影响下游3D重建可靠性，或在长距离相机运动下难以处理大范围遮挡，且需高端多GPU服务器。本文提出Genie Sim PanoWorld，一种两阶段前馈管道，通过显式可控制轨迹的全景视频连接生成与重建。通过NavMesh规划的SE(3)漫游轨迹，经密集几何扭曲条件注入潜在视频扩散模型；结合长短轨迹混合训练与基于捷径模型的自一致性目标，仅需四步无CFG去噪即可生成高保真视频。随后，前馈全景重建器将生成视频转化为支持实时自由视角漫游的高保真3D高斯场景，可直接用于具身AI模拟任务。实验表明，Genie Sim PanoWorld在全景视频生成与下游3D重建上均优于几何条件基线，并实现零样本泛化至未见室内场景。
原文摘要 · Abstract (English)
We address the problem of reconstructing a high-fidelity, freely navigable 3D scene from a single $360^\circ$ panorama, without per-scene optimization or multi-view capture. Existing methods either lack metric trajectory control, which hinders reliable downstream 3D reconstruction, or struggle with large disocclusions under long-range camera motion while requiring high-end multi-GPU servers.We present Genie Sim PanoWorld, a two-stage feed-forward pipeline that bridges generation and reconstruction via an explicit, trajectory-controllable panoramic video. A NavMesh-planned $\mathrm{SE}(3)$ roaming trajectory is injected into a latent video diffusion model through dense geometry-warped conditioning; long--short trajectory mixed training and a self-consistency objective based on shortcut models together yield high-fidelity video in four CFG-free denoising steps. A feed-forward panoramic reconstructor then lifts the generated video into a high-fidelity 3D Gaussian scene that supports real-time, free-viewpoint roaming and can be directly used as a simulation-ready asset for embodied AI applications. Experiments show that Genie Sim PanoWorld outperforms geometry-conditioned baselines in both panoramic video generation and downstream 3D reconstruction, while generalizing zero-shot to unseen indoor scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。