arXiv:2606.27978cs.CVcs.AI2026-06

提出并行滚动近似框架,让像素级自回归生成更快更准。

Parallel Rollout Approximation for Pixel-Space Autoregressive Image Generation

论文配图:Parallel Rollout Approximation for Pixel-Space Autoregressive Image Generation
图 1 · 摘自论文原文
  • 用低维中间状态替代高维像素块,实现并行训练与推理对齐。
  • 在256×256图像上达FID 1.94,超越此前百亿参数模型。
  • 兼具生成与理解能力,适合需要统一建模的视觉任务。

像素空间连续标记自回归生成直接将图像建模为原始像素块序列,避免离散标记化或独立预训练分词器。然而,其面临双重挑战:高维像素块生成导致单步误差大,教师强制训练造成训练-推理差距,使误差逐步累积。现有方法如x-预测和输入噪声注入仅部分缓解问题。精确滚动训练虽更贴近推理条件,但因串行采样过慢而难以实用。本文提出并行滚动近似(PRA)框架,联合解决上述问题。PRA生成低维中间状态而非高维像素块,再通过像素解码器映射回像素空间标记,保持像素输入-输出的自回归接口。同时,通过与推理时相同的中间状态到像素路径,独立地跨位置构建类推理的像素输入,近似推理时的像素反馈机制,同时保留并行教师强制训练。在256×256分辨率下,类条件ImageNet-1K生成任务中,参数量135M的PRA-S取得FID 2.58,优于此前百亿参数模型(FID 3.60)。扩展至511M参数的PRA-L进一步将FID降至1.94,成为像素空间自回归模型新基准。此外,PRA在图像分类探针任务中表现优于其他自回归与扩散基线,表明其在统一像素空间图像生成与理解中的潜力。

原文摘要 · Abstract (English)

Pixel-space continuous-token autoregressive (AR) generation directly models images as sequences of raw pixel patches, avoiding discrete tokenization or a separately pretrained tokenizer. However, it faces coupled challenges: high-dimensional patch generation causes large single-step errors, and teacher-forced training creates a train--inference gap that makes these errors accumulate across AR steps. Existing fixes such as $x$-prediction and input noise injection only partially mitigate these issues. Exact rollout training better matches inference-time conditions, but is impractical due to prohibitively slow sequential sampling. We propose \emph{Parallel Rollout Approximation} (PRA), a scalable framework that addresses both challenges jointly. PRA generates low-dimensional intermediate states instead of high-dimensional pixel patches, then maps them back to pixel-space tokens with a pixel decoder, preserving a pixel-in, pixel-out AR interface. It also constructs inference-like pixel inputs through the same intermediate-state-to-pixel path used at inference, independently across positions, approximating the pixel-feedback interface encountered during inference-time rollout while retaining parallel teacher-forced training. On class-conditional ImageNet-1K generation at $256\times256$ resolution, PRA-S with 135M parameters achieves an FID of 2.58, surpassing the previous billion-scale pixel-space AR result of 3.60. Scaling to PRA-L with 511M parameters further improves FID to 1.94, establishing a new state of the art among pixel-space AR models. Beyond generation, PRA achieves higher ImageNet classification probing accuracy than other AR and diffusion baselines, suggesting its potential for unified pixel-space image generation and understanding.

自回归生成像素空间图像生成并行训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。