无需微调即可生成无畸变360°全景图，解决极区失真问题。
SphereDiff: Tuning-free 360° Static and Dynamic Panorama Generation via Spherical Latent Representation
- 用球面潜在表示替代传统投影，保持各视角画质一致。
- 直接使用预训练扩散模型，生成静态与动态全景图均优于现有方法。
- 适合AR/VR内容生成，尤其关注极区质量的开发者。
随着AR/VR应用需求增长，高质量360°内容（如实时壁纸）愈发重要。然而，等距柱状投影（ERP）引入的严重畸变使高质量全景生成仍具挑战。现有方法或在有限ERP数据集上微调预训练扩散模型，或采用无需微调但依赖ERP潜在表示的方法，常导致极区出现明显畸变。本文提出SphereDiff，一种无需额外微调即可生成高质量360°静态与动态全景图的新方法。我们定义球面潜在表示，确保所有视角（包括极区）画质一致；将MultiDiffusion扩展至球面潜在空间，并提出动态球面潜在采样方法，实现预训练扩散模型的直接使用；此外引入畸变感知加权平均进一步提升生成质量。实验表明，该方法在生成360°静态与动态全景图方面显著优于现有方法，为沉浸式AR/VR应用提供稳健解决方案。代码已开源。
原文摘要 · Abstract (English)
The increasing demand for AR/VR applications has highlighted the need for high-quality content, such as 360° live wallpapers. However, generating high-quality 360° panoramic contents remains a challenging task due to the severe distortions introduced by equirectangular projection (ERP). Existing approaches either fine-tune pretrained diffusion models on limited ERP datasets or adopt tuning-free methods that still rely on ERP latent representations, often resulting in distracting distortions near the poles. In this paper, we introduce SphereDiff, a novel approach for synthesizing 360° static and live wallpaper with state-of-the-art diffusion models without additional tuning. We define a spherical latent representation that ensures consistent quality across all perspectives, including near the poles. Then, we extend MultiDiffusion to spherical latent representation and propose a dynamic spherical latent sampling method to enable direct use of pretrained diffusion models. Moreover, we introduce distortion-aware weighted averaging to further improve the generation quality. Our method outperforms existing approaches in generating 360° static and live wallpaper, making it a robust solution for immersive AR/VR applications. The code is available here. https://github.com/pmh9960/SphereDiff
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。