arXiv:2511.12181cs.CVcs.LG2025-11

用离散先验引导连续空间生成,提升图像质量与效率

MixAR: Mixture Autoregressive Image Generation

  • 用离散令牌作为先验,指导连续空间的自回归建模
  • 混合策略在保持高效的同时,显著提升图像生成质量
  • 适合追求高保真图像生成的开发者和研究者

自回归(AR)方法将图像表示为有限码本中离散标记的序列,在图像生成中取得显著成功。然而,量化过程和有限码本大小不可避免地丢失细粒度信息,限制了生成质量。受此限制启发,近期研究探索在连续潜在空间中进行自回归建模,可实现更高生成质量。但不同于受固定码本约束的离散标记,连续表示存在于广阔且无结构的空间中,给高效自回归建模带来挑战。为此,我们提出 MixAR,一种利用混合训练范式,将离散标记作为先验指导连续自回归建模的新框架。MixAR 是一种因子化形式,利用离散标记为连续自回归预测提供先验引导。我们研究了多种离散-连续混合策略,包括自注意力(DC-SA)、交叉注意力(DC-CA)以及一种简单方法(DC-Mix),后者以信息丰富的离散标记替代同质掩码标记。此外,为弥合真实训练标记与预训练自回归模型生成推理标记之间的差距,我们提出训练-推理混合(TI-Mix),实现训练与生成分布的一致性。实验表明,DC-Mix 策略在计算效率与生成保真度之间取得良好平衡,而 TI-Mix 策略带来持续改进。

原文摘要 · Abstract (English)

Autoregressive (AR) approaches, which represent images as sequences of discrete tokens from a finite codebook, have achieved remarkable success in image generation. However, the quantization process and the limited codebook size inevitably discard fine-grained information, placing bottlenecks on fidelity. Motivated by this limitation, recent studies have explored autoregressive modeling in continuous latent spaces, which offers higher generation quality. Yet, unlike discrete tokens constrained by a fixed codebook, continuous representations lie in a vast and unstructured space, posing significant challenges for efficient autoregressive modeling. To address these challenges, we introduce MixAR, a novel framework that leverages mixture training paradigms to inject discrete tokens as prior guidance for continuous AR modeling. MixAR is a factorized formulation that leverages discrete tokens as prior guidance for continuous autoregressive prediction. We investigate several discrete-continuous mixture strategies, including self-attention (DC-SA), cross-attention (DC-CA), and a simple approach (DC-Mix) that replaces homogeneous mask tokens with informative discrete counterparts. Moreover, to bridge the gap between ground-truth training tokens and inference tokens produced by the pre-trained AR model, we propose Training-Inference Mixture (TI-Mix) to achieve consistent training and generation distributions. In our experiments, we demonstrate a favorable balance of the DC-Mix strategy between computational efficiency and generation fidelity, and consistent improvement of TI-Mix.

图像生成自回归模型连续潜空间混合建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。