统一架构实现图文生成与编辑，1024×1024图像5秒生成。
NextFlow: Unified Sequential Modeling Activates Multimodal Understanding and Generation
- 用统一自回归框架，文本用词预测，图像用尺度预测。
- 1024×1024图像生成仅需5秒，比同类模型快多个数量级。
- 支持图文混排、视频生成，适合多模态内容创作场景。
我们提出NextFlow，一个在6万亿交错文本-图像离散标记上训练的统一解码器仅自回归Transformer。通过在统一自回归架构中使用统一视觉表示，NextFlow原生激活多模态理解与生成能力，实现图像编辑、交错内容生成和视频生成。受模态差异启发——文本严格顺序，图像固有分层——对文本保留逐词预测，对视觉生成采用逐尺度预测。这突破了传统栅格扫描方法，使1024×1024图像生成仅需5秒，远超同类自回归模型。通过稳健的训练方案解决多尺度生成不稳定性。此外，引入前缀调优策略用于强化学习。实验表明，NextFlow在统一模型中达到最先进性能，视觉质量媲美专用扩散基线。
原文摘要 · Abstract (English)
We present NextFlow, a unified decoder-only autoregressive transformer trained on 6 trillion interleaved text-image discrete tokens. By leveraging a unified vision representation within a unified autoregressive architecture, NextFlow natively activates multimodal understanding and generation capabilities, unlocking abilities of image editing, interleaved content and video generation. Motivated by the distinct nature of modalities - where text is strictly sequential and images are inherently hierarchical - we retain next-token prediction for text but adopt next-scale prediction for visual generation. This departs from traditional raster-scan methods, enabling the generation of 1024x1024 images in just 5 seconds - orders of magnitude faster than comparable AR models. We address the instabilities of multi-scale generation through a robust training recipe. Furthermore, we introduce a prefix-tuning strategy for reinforcement learning. Experiments demonstrate that NextFlow achieves state-of-the-art performance among unified models and rivals specialized diffusion baselines in visual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。