从俯视图生成逼真全景图，解决室内结构与光影一致性难题
Top2Pano: Learning to Generate Indoor Panoramas from Top-Down View
- 先估体积占用重建3D结构，再用扩散模型精修细节
- 在两个数据集上优于基线，准确还原空间布局与遮挡关系
- 适合虚拟现实、室内设计等需快速生成沉浸式场景的领域
从二维俯视图生成沉浸式360°室内全景图在虚拟现实、室内设计、房地产和机器人领域有广泛应用。该任务因缺乏显式三维结构，且需保持几何一致性和照片级真实感而具有挑战性。本文提出Top2Pano，一种端到端模型，通过估计体素占用来推断三维结构，再利用体素渲染生成粗略的颜色与深度全景图。这些结果引导基于扩散模型的ControlNet精修阶段，提升真实感与结构保真度。在两个数据集上的评估显示，Top2Pano优于现有基线方法，能有效重建几何结构、遮挡关系和空间布局。该模型还具备良好泛化能力，可从示意图式平面图生成高质量全景图。结果表明，Top2Pano在连接俯视图与沉浸式室内合成之间具有巨大潜力。
原文摘要 · Abstract (English)
Generating immersive 360° indoor panoramas from 2D top-down views has applications in virtual reality, interior design, real estate, and robotics. This task is challenging due to the lack of explicit 3D structure and the need for geometric consistency and photorealism. We propose Top2Pano, an end-to-end model for synthesizing realistic indoor panoramas from top-down views. Our method estimates volumetric occupancy to infer 3D structures, then uses volumetric rendering to generate coarse color and depth panoramas. These guide a diffusion-based refinement stage using ControlNet, enhancing realism and structural fidelity. Evaluations on two datasets show Top2Pano outperforms baselines, effectively reconstructing geometry, occlusions, and spatial arrangements. It also generalizes well, producing high-quality panoramas from schematic floorplans. Our results highlight Top2Pano's potential in bridging top-down views with immersive indoor synthesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。