arXiv:2409.08397cs.CVcs.AI2024-09中稿 · WACV 2025, Project…被引 4

无需训练,实现全景图间无缝文本驱动转换

360PanT: Training-Free Text-Driven 360-Degree Panorama-to-Panorama Translation

  • 通过扩展输入图像嵌入边界连续性信息到噪声潜空间
  • 利用空间控制实现左右一致的无缝拼贴生成
  • 适合虚拟现实、全景内容创作等需要沉浸体验的场景

现有文本驱动的全景图到全景图图像转换方法在保持边界连续性方面仍面临挑战,常导致转换后全景图边界出现视觉突兀断裂,破坏沉浸感。为此,本文提出360PanT,一种无需训练的文本驱动360度全景图到全景图转换方法,可实现边界连续的无缝转换。该方法包含两个关键组件:边界连续性编码与带空间控制的无缝拼贴生成。首先,边界连续性编码通过构建扩展输入图像,将输入全景图的关键边界连续性信息嵌入噪声潜表示中;其次,基于该嵌入的噪声潜表示并受目标提示引导,带空间控制的无缝拼贴生成能生成左右半部分完全一致的转换图像,同时保留扩展输入的结构与语义布局。该过程确保最终生成的360度全景图具有无缝边界连续性。在真实世界与合成数据集上的实验结果表明,360PanT在全景图转换任务中有效。代码已开源。

原文摘要 · Abstract (English)

Preserving boundary continuity in the translation of 360-degree panoramas remains a significant challenge for existing text-driven image-to-image translation methods. These methods often produce visually jarring discontinuities at the translated panorama's boundaries, disrupting the immersive experience. To address this issue, we propose 360PanT, a training-free approach to text-based 360-degree panorama-to-panorama translation with boundary continuity. Our 360PanT achieves seamless translations through two key components: boundary continuity encoding and seamless tiling translation with spatial control. Firstly, the boundary continuity encoding embeds critical boundary continuity information of the input 360-degree panorama into the noisy latent representation by constructing an extended input image. Secondly, leveraging this embedded noisy latent representation and guided by a target prompt, the seamless tiling translation with spatial control enables the generation of a translated image with identical left and right halves while adhering to the extended input's structure and semantic layout. This process ensures a final translated 360-degree panorama with seamless boundary continuity. Experimental results on both real-world and synthesized datasets demonstrate the effectiveness of our 360PanT in translating 360-degree panoramas. Code is available at \href{https://github.com/littlewhitesea/360PanT}{https://github.com/littlewhitesea/360PanT}.

全景生成文本生成边界连续

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。