用自回归模型生成无限长且连贯的全景图，突破传统方法限制。
PanoLlama: Generating Endless and Coherent Panoramas with Next-Token-Prediction LLMs
- 采用自回归框架与令牌重定向策略，实现无边界全景生成。
- 在连贯性、保真度和美学上达到47.50%、28.16%、15%的SOTA表现。
- 支持无遮罩布局控制、多尺度合成，适合创意设计与虚拟场景构建。
全景图像生成（PIG）旨在创建任意长度的连贯图像。现有方法多基于联合扩散范式，但其复杂的裁剪连接设计常导致多层级连贯性不足。本文将该问题分解为核心组件，发现其天然契合下一令牌预测，因而采用自回归（AR）范式建模PIG。然而，现有视觉自回归（VAR）模型仅支持固定尺寸生成，无法处理全景图像。为此，提出PanoLlama框架，通过无需训练的令牌重定向策略，突破现有模型尺寸限制，实现水平与垂直方向的下一区域预测，重构了PIG流程。该方法在连贯性（47.50%）、保真度（28.16%）和美学（15%）上达到当前最优性能，并支持无遮罩布局控制、多尺度与多引导合成等新应用。为促进标准化评估，还构建了包含1000个提示、覆盖100+主题的数据集，为PIG研究提供新基准。代码已开源。
原文摘要 · Abstract (English)
Panoramic Image Generation (PIG) aims to create coherent images of arbitrary lengths. Most existing methods fall in the joint diffusion paradigm, but their complex and heuristic crop connection designs often limit their ability to achieve multilevel coherence. By deconstructing this challenge into its core components, we find it naturally aligns with next-token prediction, leading us to adopt an autoregressive (AR) paradigm for PIG modeling. However, existing visual AR (VAR) models are limited to fixed-size generation, lacking the capability to produce panoramic images. In this paper, we propose PanoLlama, a novel framework that achieves endless and coherent panorama generation with the autoregressive paradigm. Our approach develops a training-free strategy that utilizes token redirection to overcome the size limitations of existing VAR models, enabling next-crop prediction in both horizontal and vertical directions. This refreshes the PIG pipeline while achieving SOTA performance in coherence (47.50%), fidelity(28.16%), and aesthetics (15%). Additionally, PanoLlama supports applications other PIG methods cannot achieve, including mask-free layout control, multi-scale and multi-guidance synthesis. To facilitate standardized evaluation, we also establish a dataset with 1,000 prompts spanning 100+ themes, providing a new testing benchmark for PIG research. The code is available at https://github.com/0606zt/PanoLlama.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。