让图像生成顺序随意，打破固定顺序限制。
RandAR: Decoder-only Autoregressive Visual Generation in Random Orders
- 用位置指令令牌指定生成顺序，实现任意顺序生成
- 在随机顺序训练下性能接近传统扫描顺序模型
- 支持零样本修复、扩图和超分辨率，推理提速2.5倍
我们提出 RandAR，一种无需预设生成顺序的解码器仅视觉自回归模型。通过在每个待预测图像标记前插入表示空间位置的“位置指令标记”，模型可自由选择生成顺序。在随机排列标记序列上训练，虽更具挑战性，但性能与传统栅格顺序模型相当。更重要的是，从随机顺序训练的解码器仅变压器模型获得了新能力：推理时采用带键值缓存的并行解码，实现2.5倍加速且不损失质量；同时支持零样本图像修复、外扩和分辨率外推。我们希望 RandAR 能为解码器仅视觉生成开辟新方向，并拓展其在多样场景中的应用。项目主页见 https://rand-ar.github.io/。
原文摘要 · Abstract (English)
We introduce RandAR, a decoder-only visual autoregressive (AR) model capable of generating images in arbitrary token orders. Unlike previous decoder-only AR models that rely on a predefined generation order, RandAR removes this inductive bias, unlocking new capabilities in decoder-only generation. Our essential design enables random order by inserting a "position instruction token" before each image token to be predicted, representing the spatial location of the next image token. Trained on randomly permuted token sequences -- a more challenging task than fixed-order generation, RandAR achieves comparable performance to its conventional raster-order counterpart. More importantly, decoder-only transformers trained from random orders acquire new capabilities. For the efficiency bottleneck of AR models, RandAR adopts parallel decoding with KV-Cache at inference time, enjoying 2.5x acceleration without sacrificing generation quality. Additionally, RandAR supports inpainting, outpainting and resolution extrapolation in a zero-shot manner. We hope RandAR inspires new directions for decoder-only visual generation models and broadens their applications across diverse scenarios. Our project page is at https://rand-ar.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。