arXiv:2506.21681cs.CVcs.LG2025-06被引 3

用切平面扩散模型生成高质量全景图,支持复杂文本提示。

TanDiT: Tangent-Plane Diffusion Transformer for High-Quality 360° Panorama Generation

  • 通过单次去噪迭代生成覆盖360°的切平面图像网格。
  • 在多个数据集上实现高保真度与无缝拼接,优于现有方法。
  • 适合需要高质量全景生成的应用,如虚拟现实、数字孪生。

近期图像生成技术在透视图像合成方面取得显著进展,但全景图像生成仍面临几何畸变程度不一和无缝环形一致性要求等挑战。为应对这些问题并发挥现有模型优势,我们提出TanDiT,一种通过生成覆盖整个360°视域的切平面图像网格来合成全景场景的方法。不同于依赖多个扩散分支的先前方法,TanDiT采用统一扩散模型,在单次去噪迭代中同时生成这些切平面图像。此外,我们提出一种模型无关的后处理步骤,专门用于增强生成全景图的全局一致性。为准确评估全景图像质量,我们还引入两个专用指标:TangentIS和TangentFID,并提供包含带标注全景数据集及标准化评估脚本的综合基准。大量实验表明,该方法能有效泛化至训练数据之外,稳健解析细节丰富且复杂的文本提示,并可与多种生成模型无缝集成,产出高质量、多样化的全景图像。

原文摘要 · Abstract (English)

Recent advances in image generation have led to remarkable improvements in synthesizing perspective images. However, these models still struggle with panoramic image generation due to unique challenges, including varying levels of geometric distortion and the requirement for seamless loop-consistency. To address these issues while leveraging the strengths of the existing models, we introduce TanDiT, a method that synthesizes panoramic scenes by generating grids of tangent-plane images covering the entire 360$^\circ$ view. Unlike previous methods relying on multiple diffusion branches, TanDiT utilizes a unified diffusion model trained to produce these tangent-plane images simultaneously within a single denoising iteration. Furthermore, we propose a model-agnostic post-processing step specifically designed to enhance global coherence across the generated panoramas. To accurately assess panoramic image quality, we also present two specialized metrics, TangentIS and TangentFID, and provide a comprehensive benchmark comprising captioned panoramic datasets and standardized evaluation scripts. Extensive experiments demonstrate that our method generalizes effectively beyond its training data, robustly interprets detailed and complex text prompts, and seamlessly integrates with various generative models to yield high-quality, diverse panoramic images.

全景生成扩散模型切平面图像合成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。