arXiv:2503.10696cs.CVeess.IV2025-03ICCV被引 29

新方法让图像视频生成更快更准,靠的是从中心向外逐步预测邻近像素。

Neighboring Autoregressive Modeling for Efficient Visual Generation

  • 从中心出发,按曼哈顿距离顺序预测邻近像素,实现渐进式生成。
  • 图像和视频生成速度提升2.4到8.6倍,质量优于现有方法。
  • 适合追求高效高质生成的开发者与研究者使用。

视觉自回归模型通常采用光栅顺序的“下一个标记预测”范式,忽略了视觉内容中固有的空间与时间局部性。事实上,视觉标记与其空间或时间上相邻的标记具有更强的相关性。本文提出邻近自回归建模(NAR),将视觉生成重构为一种逐步向外扩展的出图过程,遵循“由近及远”的“下一邻域预测”机制。从一个初始标记开始,其余标记按其在时空空间中相对于初始标记的曼哈顿距离递增顺序解码,逐步扩大已解码区域边界。为支持在时空空间中并行预测多个邻近标记,我们引入一组方向导向的解码头,每个头沿相互正交的方向预测下一个标记。推理时,所有与已解码标记相邻的标记可并行处理,显著减少生成所需的前向传播步数。在ImageNet 256×256和UCF101上的实验表明,NAR分别实现了2.4倍和8.6倍的吞吐量提升,且在图像与视频生成任务中均获得更优的FID/FVD得分,优于PAR-4X方法。在文本到图像生成基准GenEval上,参数量仅0.8B的NAR超越了参数量7B的Chameleon,且仅使用其40%的训练数据。代码已公开于https://github.com/ThisisBillhe/NAR。

原文摘要 · Abstract (English)

Visual autoregressive models typically adhere to a raster-order ``next-token prediction" paradigm, which overlooks the spatial and temporal locality inherent in visual content. Specifically, visual tokens exhibit significantly stronger correlations with their spatially or temporally adjacent tokens compared to those that are distant. In this paper, we propose Neighboring Autoregressive Modeling (NAR), a novel paradigm that formulates autoregressive visual generation as a progressive outpainting procedure, following a near-to-far ``next-neighbor prediction" mechanism. Starting from an initial token, the remaining tokens are decoded in ascending order of their Manhattan distance from the initial token in the spatial-temporal space, progressively expanding the boundary of the decoded region. To enable parallel prediction of multiple adjacent tokens in the spatial-temporal space, we introduce a set of dimension-oriented decoding heads, each predicting the next token along a mutually orthogonal dimension. During inference, all tokens adjacent to the decoded tokens are processed in parallel, substantially reducing the model forward steps for generation. Experiments on ImageNet$256\times 256$ and UCF101 demonstrate that NAR achieves 2.4$\times$ and 8.6$\times$ higher throughput respectively, while obtaining superior FID/FVD scores for both image and video generation tasks compared to the PAR-4X approach. When evaluating on text-to-image generation benchmark GenEval, NAR with 0.8B parameters outperforms Chameleon-7B while using merely 0.4 of the training data. Code is available at https://github.com/ThisisBillhe/NAR.

自回归生成图像生成高效推理扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。