arXiv:2503.18420cs.CV2025-03CVPR被引 15

提出新方法生成无畸变的全景图,解决旧方法‘视觉欺骗’问题。

Panorama Generation From NFoV Image Done Right

  • 分步生成:先控制畸变,再补全内容
  • 新指标Distort-CLIP让模型更关注畸变精度
  • 适合做高质量虚拟现实全景生成的开发者

从窄视场图像生成360度全景图是元宇宙应用的重要任务。现有方法多用InceptionNet或CLIP评估,但这些指标不擅长衡量畸变。本文首次提出专用于畸变评估的Distort-CLIP,并发现此前方法存在‘视觉欺骗’现象——为提升画面美观而牺牲畸变准确性。这源于模型同时学习畸变与内容补全,导致更倾向优化后者。为此,本文提出PanoDecouple框架,将生成过程解耦:通过引入具有全景畸变先验的DistortNet进行畸变引导,结合视角图像信息的ContentNet完成内容补全,并设计基于Distort-CLIP的畸变纠正损失函数。实验表明,PanoDecouple在畸变和视觉质量上均优于现有方法。

原文摘要 · Abstract (English)

Generating 360-degree panoramas from narrow field of view (NFoV) image is a promising computer vision task for Virtual Reality (VR) applications. Existing methods mostly assess the generated panoramas with InceptionNet or CLIP based metrics, which tend to perceive the image quality and is \textbf{not suitable for evaluating the distortion}. In this work, we first propose a distortion-specific CLIP, named Distort-CLIP to accurately evaluate the panorama distortion and discover the \textbf{``visual cheating''} phenomenon in previous works (\ie, tending to improve the visual results by sacrificing distortion accuracy). This phenomenon arises because prior methods employ a single network to learn the distinct panorama distortion and content completion at once, which leads the model to prioritize optimizing the latter. To address the phenomenon, we propose \textbf{PanoDecouple}, a decoupled diffusion model framework, which decouples the panorama generation into distortion guidance and content completion, aiming to generate panoramas with both accurate distortion and visual appeal. Specifically, we design a DistortNet for distortion guidance by imposing panorama-specific distortion prior and a modified condition registration mechanism; and a ContentNet for content completion by imposing perspective image information. Additionally, a distortion correction loss function with Distort-CLIP is introduced to constrain the distortion explicitly. The extensive experiments validate that PanoDecouple surpasses existing methods both in distortion and visual metrics.

全景生成扩散模型畸变控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。