arXiv:2607.00832cs.CVcs.AI2026-07被引 3

单张全景图生成可自由漫游的3D场景,无需多步迭代。

Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences

论文配图:Pano2World: End-to-End 3D Generation via Unified Multi-View Sequences
图 1 · 摘自论文原文
  • 用自适应采样视角重建粗略3D代理,生成几何对齐引导图。
  • 通过视图感知注意力联合去噪所有目标视角,实现跨视角一致性。
  • 提出隐空间适配器直接提炼隐特征,避免解码信息损失。

单张全景图虽能捕捉完整视觉球面,但仅限原地环视,无法实现真实场景探索。将单张全景图转换为可持久渲染、支持自由视角导航的3D表示受到广泛关注。现有方法或采用逐视图迭代补全,导致误差累积和复杂多步流程;或依赖视频生成模型的时间一致性先验,但连续轨迹约束限制了多方向同时覆盖的能力。本文提出Pano2World,输入单张室内全景图,直接输出可探索的3D高斯场景。首先基于源全景图重建粗略3D高斯代理,并在自适应采样的邻近视角渲染生成几何对齐的引导全景图;随后,全景扩散模型通过视图感知注意力路由,联合去噪所有目标视图,每个目标视图同时接收对应引导图的几何约束与源图的全局语义引导,自然实现跨视角一致性。为避免联合去噪过程中多视图隐特征经VAE解码回像素域带来的信息损失,引入几何感知的隐空间适配器(Latent Feature Adapter),直接将这些隐特征提炼为场景隐变量,再解码为最终3D高斯场景。实验表明,Pano2World在多位置全景新视角合成基准上显著优于现有方法。

原文摘要 · Abstract (English)

A single panorama captures the full visual sphere from one camera center, yet confines users to looking around in place without enabling true scene exploration. Converting a single panorama into a persistent, renderable 3D representation for free-viewpoint navigation has attracted growing interest; existing methods either adopt iterative per-view completion that propagates inpainting results to update the underlying geometry, leading to progressive error accumulation and cumbersome multi-step pipelines, or leverage the temporal consistency priors of video generation models, yet the continuous-trajectory constraint intrinsic to such models limits their flexibility in covering scenes from multiple directions simultaneously. We present Pano2World, which takes a single indoor panorama as input and directly outputs a persistent, explorable 3D Gaussian scene. Given the source panorama, Pano2World first reconstructs a coarse 3D Gaussian proxy and renders it at adaptively sampled nearby poses to obtain geometrically aligned guidance panoramas; a panoramic diffusion model then jointly denoises all target views via View-Aware Attention Routing, where each target view simultaneously receives geometric constraints from its corresponding guidance panorama and global semantic guidance from the source panorama, naturally enforcing cross-view consistency. To avoid the information loss incurred by decoding the multi-view hidden features formed during joint denoising back to the pixel domain via VAE, we introduce Latent Feature Adapter, a geometry-aware bridge module that directly distills these hidden features into a scene latent, subsequently decoded into the final 3D Gaussian scene. Experiments demonstrate that Pano2World significantly outperforms existing methods on the multi-position panoramic novel-view synthesis benchmark.

3D生成全景图扩散模型高斯渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。