让扩散模型适应全景图修复,解决扭曲与接缝问题。
Gimbal360: Canonicalizing Planar Diffusion for Spherical Panorama Completion
- 构建统一视图空间,消除视角扭曲对齐全景图
- 实现无缝拼接,修复后全景图无接缝、视觉质量优
- 适合做全景图像生成与修复的研究者和开发者
扩散模型在二维图像修复中表现强大,但其先验基于有限平面图像,难以直接应用于360度全景图。透视图与球面全景图在投影几何与拓扑结构上存在差异:视角依赖的畸变使空间对应复杂,而等距圆柱投影(ERP)全景图具有内在的$S^1$周期性,标准欧氏架构无法保持。本文提出Gimbal360,通过标准化几何与拓扑结构,将平面扩散先验适配至球面全景图修复。其标准视图空间将投影畸变表示为纬度的固定函数,实现透视输入与球面全景图间的稳定映射。为将无标注的自然图像映射到该空间,提出可微投影归一化,在无需相机参数的情况下将密集对应场投影至三维刚性投影流形。此外,引入拓扑等变生成机制,强制潜在空间平移等变性以保持ERP边界处的连续性。这些设计使扩散模型在显式匹配球面域的表示中运行。同时构建了Horizon360,一个重力对齐的大规模全景环境数据集。大量实验表明,Gimbal360在360度场景修复中达到当前最优的视觉保真度与接缝连续性。
原文摘要 · Abstract (English)
Diffusion models provide powerful priors for 2D image completion, but these priors are learned on bounded planar images and do not transfer directly to $360^\circ$ panoramas. Perspective observations and spherical panoramas differ in both projective geometry and topology: viewpoint-dependent distortion complicates spatial correspondence, while Equirectangular Projection (ERP) panoramas exhibit intrinsic $S^1$ periodicity that standard Euclidean architectures do not preserve. We present Gimbal360, a unified framework that adapts planar diffusion priors to spherical panoramic completion by standardizing these geometric and topological structures. Our Canonical Viewing Space expresses projective distortion as a fixed function of latitude, providing a consistent interface between perspective inputs and spherical panoramas. To map unposed in-the-wild images into this space, Differentiable Projective Canonicalization projects a dense correspondence field onto a 3-DoF rigid projection manifold without requiring camera parameters at inference. We further introduce Topologically Equivariant Generation, which enforces latent shift equivariance to preserve continuity across the periodic ERP boundary. Together, these designs allow diffusion to operate in a representation whose geometry and topology are explicitly aligned with the spherical domain. We also introduce Horizon360, a curated large-scale dataset of gravity-aligned panoramic environments. Extensive experiments show that Gimbal360 achieves state-of-the-art visual fidelity and seam continuity in $360^\circ$ scene completion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。