通过分层嵌套自回归结构，实现高效高多样性图像生成
Nested AutoRegressive Models
- 设计分层嵌套自回归模块，大尺度模块依赖小尺度输出
- 生成复杂度从O(n)降至O(log n)，计算效率显著提升
- 适合追求高效图像生成的开发者与研究者
自回归（AR）模型在图像生成中表现优异，性能接近扩散模型。但其逐标记生成机制计算开销大，现有方法如VAR又限制了样本多样性。本文提出嵌套自回归模型（NestAR），采用分层多尺度架构，较大尺度模块基于前级小尺度模块输出进行条件生成；每层内使用自回归结构生成标记块。该嵌套架构将生成复杂度从O(n)降至O(log n)，同时提升图像多样性。NestAR引入流匹配损失以使用连续标记，并设计协调多尺度模块的训练目标。实验表明，模型在保持竞争力图像生成性能的同时，显著降低计算成本。
原文摘要 · Abstract (English)
AutoRegressive (AR) models have demonstrated competitive performance in image generation, achieving results comparable to those of diffusion models. However, their token-by-token image generation mechanism remains computationally intensive and existing solutions such as VAR often lead to limited sample diversity. In this work, we propose a Nested AutoRegressive~(NestAR) model, which proposes nested AutoRegressive architectures in generating images. NestAR designs multi-scale modules in a hierarchical order. These different scaled modules are constructed in an AR architecture, where one larger-scale module is conditioned on outputs from its previous smaller-scale module. Within each module, NestAR uses another AR structure to generate ``patches'' of tokens. The proposed nested AR architecture reduces the overall complexity from $\mathcal{O}(n)$ to $\mathcal{O}(\log n)$ in generating $n$ image tokens, as well as increases image diversities. NestAR further incorporates flow matching loss to use continuous tokens, and develops objectives to coordinate these multi-scale modules in model training. NestAR achieves competitive image generation performance while significantly lowering computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。